2024-03-11T18:55:14.426 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-11T18:55:14.427 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259 --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/orig.config.yaml 2024-03-11T18:55:14.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T18:55:14.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591259' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 94a539707733325322964c6a187c6a1b41be1daf subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVqXqWtJm6fZ0pnTaM1i7DIeC4Ww+DgoOBY7RfyaD6rQf2472fJIfu/PWrQTXXT0O5OYc0TqExv2cqAoTYzqHY= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOc0GKDBSBvCA0swtD/8Mb7zanz47X4xx7vSMex+ndErKYfZQACUTyvU9TlhmL3bA95OTI3JkFpUUpBHIgPpSQw= smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE+UwAq37y0s5vpPONSzDIQm2UPZ6ctj6LvDzxHcSvsZGm9kbLzQx9eizAdRULCRXTQ0qYQiZRC91LVDv2g7zRs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-11T18:55:14.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-11T18:55:14.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-11T18:55:14.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-11T18:55:14.593 INFO:teuthology.task.internal:Checking packages... 2024-03-11T18:55:14.620 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-11T18:55:14.620 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-11T18:55:14.620 INFO:teuthology.packaging:ref: None 2024-03-11T18:55:14.620 INFO:teuthology.packaging:tag: None 2024-03-11T18:55:14.621 INFO:teuthology.packaging:branch: squid 2024-03-11T18:55:14.621 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:55:14.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-11T18:55:14.852 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607.g94a53970 2024-03-11T18:55:14.853 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-11T18:55:14.862 INFO:teuthology.task.internal:no buildpackages task found 2024-03-11T18:55:14.862 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-11T18:55:14.888 INFO:teuthology.task.internal:Saving configuration 2024-03-11T18:55:14.909 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-11T18:55:14.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T18:55:14.942 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.573305', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVqXqWtJm6fZ0pnTaM1i7DIeC4Ww+DgoOBY7RfyaD6rQf2472fJIfu/PWrQTXXT0O5OYc0TqExv2cqAoTYzqHY='} 2024-03-11T18:55:14.964 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.574642', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOc0GKDBSBvCA0swtD/8Mb7zanz47X4xx7vSMex+ndErKYfZQACUTyvU9TlhmL3bA95OTI3JkFpUUpBHIgPpSQw='} 2024-03-11T18:55:14.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.571727', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE+UwAq37y0s5vpPONSzDIQm2UPZ6ctj6LvDzxHcSvsZGm9kbLzQx9eizAdRULCRXTQ0qYQiZRC91LVDv2g7zRs='} 2024-03-11T18:55:14.990 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-11T18:55:15.001 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-11T18:55:15.001 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-11T18:55:15.001 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-11T18:55:15.002 INFO:teuthology.run_tasks:Running task console_log... 2024-03-11T18:55:15.147 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0582f6b160>, signals=[15]) 2024-03-11T18:55:15.148 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-11T18:55:15.160 INFO:teuthology.task.internal:Opening connections... 2024-03-11T18:55:15.161 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T18:55:15.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:55:15.302 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-03-11T18:55:15.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:55:15.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-03-11T18:55:15.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:55:15.503 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-11T18:55:15.513 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-03-11T18:55:15.531 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-03-11T18:55:15.531 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T18:55:15.589 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T18:55:15.590 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-03-11T18:55:15.614 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-03-11T18:55:15.632 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-03-11T18:55:15.632 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-03-11T18:55:15.689 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-03-11T18:55:15.689 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-03-11T18:55:15.689 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-03-11T18:55:15.689 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T18:55:15.690 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T18:55:15.691 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-03-11T18:55:15.714 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-03-11T18:55:15.732 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-03-11T18:55:15.732 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-03-11T18:55:15.789 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T18:55:15.790 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T18:55:15.790 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-03-11T18:55:15.814 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-11T18:55:15.826 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-11T18:55:15.846 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-11T18:55:15.846 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-03-11T18:55:15.849 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-03-11T18:55:15.851 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-03-11T18:55:15.868 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-11T18:55:15.882 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-11T18:55:15.882 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-03-11T18:55:15.907 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-03-11T18:55:15.910 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-03-11T18:55:15.942 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-11T18:55:16.033 INFO:teuthology.run_tasks:Running task kernel... 2024-03-11T18:55:16.050 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-11T18:55:16.050 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-11T18:55:16.050 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-11T18:55:16.051 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T18:55:16.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:55:16.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-11T18:55:16.431 DEBUG:teuthology.task.kernel:sha1 for client.0 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T18:55:16.431 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-11T18:55:16.432 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-03-11T18:55:16.450 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-361.el9.x86_64 2024-03-11T18:55:16.450 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T18:55:16.451 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-11T18:55:16.452 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum upgrade -y linux-firmware 2024-03-11T18:55:19.658 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-11T18:55:21.043 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-03-11T18:55:22.111 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-11T18:55:26.722 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-03-11T18:55:36.832 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 943 kB/s | 6.1 MB 00:06 2024-03-11T18:55:38.442 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-11T18:55:40.699 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 21 MB 00:02 2024-03-11T18:55:45.547 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-11T18:55:47.853 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-03-11T18:55:47.867 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-03-11T18:55:47.867 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-03-11T18:55:47.867 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-03-11T18:55:47.867 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 1 Package 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 375 M 2024-03-11T18:55:47.868 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-03-11T18:56:03.809 INFO:teuthology.orchestra.run.smithi072.stdout:linux-firmware-20240219-143.el9.noarch.rpm 24 MB/s | 375 MB 00:15 2024-03-11T18:56:03.810 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-03-11T18:56:03.811 INFO:teuthology.orchestra.run.smithi072.stdout:Total 24 MB/s | 375 MB 00:15 2024-03-11T18:56:03.811 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-03-11T18:56:03.915 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-03-11T18:56:03.915 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-03-11T18:56:06.305 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-03-11T18:56:06.306 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-03-11T18:56:14.583 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-03-11T18:56:14.813 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-11T18:56:14.913 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T18:56:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T18:56:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-11T18:56:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T18:56:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:56:17.062 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-03-11T18:56:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-11T18:56:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:56:17.063 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-03-11T18:56:17.420 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-11T18:56:17.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T18:56:17.642 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-11T18:56:17.642 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-11T18:56:17.645 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi072.front.sepia.ceph.com to complete... 2024-03-11T18:56:29.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11 18:56:29 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-11T18:56:29.083 INFO:teuthology.orchestra.run.smithi072.stderr:FINISHED --2024-03-11 18:56:29-- 2024-03-11T18:56:29.083 INFO:teuthology.orchestra.run.smithi072.stderr:Total wall clock time: 11s 2024-03-11T18:56:29.083 INFO:teuthology.orchestra.run.smithi072.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-11T18:56:29.087 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-11T18:56:29.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-11T18:56:31.012 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying... ######################################## 2024-03-11T18:56:31.083 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing... ######################################## 2024-03-11T18:56:31.161 INFO:teuthology.orchestra.run.smithi072.stdout:Updating / installing... 2024-03-11T18:56:38.615 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-11T18:57:09.264 INFO:teuthology.orchestra.run.smithi072.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-11T18:57:33.028 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-11T18:57:33.028 DEBUG:teuthology.orchestra.run.smithi072:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-11T18:57:33.112 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-11T18:57:33.113 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-11T18:57:33.114 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-11T18:57:33.115 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-11T18:57:33.116 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-11T18:57:33.117 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-11T18:57:33.118 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-11T18:57:33.119 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-11T18:57:33.120 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-11T18:57:33.121 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-11T18:57:33.122 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-11T18:57:33.123 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-11T18:57:33.124 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-11T18:57:33.125 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-11T18:57:33.126 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-11T18:57:33.127 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-11T18:57:33.128 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-11T18:57:33.129 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-11T18:57:33.130 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-11T18:57:33.131 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-11T18:57:33.132 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-11T18:57:33.133 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-11T18:57:33.134 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-11T18:57:33.135 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-11T18:57:33.136 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-11T18:57:33.137 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-11T18:57:33.138 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-11T18:57:33.139 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-11T18:57:33.140 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-11T18:57:33.141 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-11T18:57:33.142 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-11T18:57:33.143 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-11T18:57:33.144 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-11T18:57:33.145 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-11T18:57:33.146 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-11T18:57:33.147 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-11T18:57:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-11T18:57:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-11T18:57:33.150 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-11T18:57:33.151 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-11T18:57:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-11T18:57:33.153 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-11T18:57:33.154 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-11T18:57:33.155 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-11T18:57:33.156 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-11T18:57:33.157 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-11T18:57:33.158 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-11T18:57:33.159 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-11T18:57:33.160 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-11T18:57:33.161 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-11T18:57:33.162 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-11T18:57:33.163 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-11T18:57:33.164 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-11T18:57:33.165 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-11T18:57:33.166 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-11T18:57:33.167 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-11T18:57:33.168 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-11T18:57:33.169 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-11T18:57:33.170 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-11T18:57:33.171 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-11T18:57:33.172 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-11T18:57:33.173 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-11T18:57:33.174 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-11T18:57:33.175 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-11T18:57:33.176 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-11T18:57:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-11T18:57:33.178 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-11T18:57:33.179 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-11T18:57:33.180 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-11T18:57:33.181 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-11T18:57:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-11T18:57:33.183 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-11T18:57:33.184 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-11T18:57:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-11T18:57:33.186 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-11T18:57:33.187 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-11T18:57:33.188 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-11T18:57:33.189 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-11T18:57:33.190 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-11T18:57:33.191 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-11T18:57:33.192 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-11T18:57:33.193 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-11T18:57:33.194 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-11T18:57:33.195 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-11T18:57:33.196 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-11T18:57:33.197 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-11T18:57:33.198 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-11T18:57:33.199 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-11T18:57:33.200 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-11T18:57:33.201 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-11T18:57:33.202 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-11T18:57:33.203 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-11T18:57:33.204 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-11T18:57:33.205 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-11T18:57:33.206 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-11T18:57:33.207 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-11T18:57:33.208 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-11T18:57:33.209 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-11T18:57:33.210 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-11T18:57:33.211 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-11T18:57:33.212 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-11T18:57:33.213 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-11T18:57:33.214 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-11T18:57:33.215 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-11T18:57:33.216 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-11T18:57:33.217 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-11T18:57:33.218 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-11T18:57:33.219 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-11T18:57:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-11T18:57:33.221 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-11T18:57:33.222 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-11T18:57:33.223 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-11T18:57:33.224 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-11T18:57:33.225 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-11T18:57:33.226 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-11T18:57:33.227 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-11T18:57:33.228 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-11T18:57:33.229 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-11T18:57:33.230 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-11T18:57:33.231 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-11T18:57:33.232 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-11T18:57:33.233 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-11T18:57:33.234 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-11T18:57:33.235 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-11T18:57:33.236 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-11T18:57:33.237 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-11T18:57:33.238 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-11T18:57:33.239 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-11T18:57:33.240 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-11T18:57:33.241 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-11T18:57:33.242 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-11T18:57:33.243 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-11T18:57:33.244 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-11T18:57:33.245 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-11T18:57:33.250 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:33.250 DEBUG:teuthology.orchestra.run.smithi072:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-11T18:57:33.272 INFO:teuthology.orchestra.run.smithi072.stdout:postinstall scriptlet (using /bin/sh): 2024-03-11T18:57:33.272 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-11T18:57:33.272 INFO:teuthology.orchestra.run.smithi072.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:for file in vmlinuz System.map config; do 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout: fi 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:done 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-11T18:57:33.273 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:33.274 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-03-11T18:57:33.274 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2024-03-11T18:57:33.348 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.06 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 68.el9 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Group : Unspecified 2024-03-11T18:57:33.349 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 8264254 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : builder@centos.org 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2024-03-11T18:57:33.350 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T18:57:33.351 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2024-03-11T18:57:33.353 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-11T18:57:33.353 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:33.353 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-11T18:57:33.977 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2024-03-11T18:57:35.537 INFO:teuthology.orchestra.run.smithi072.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-11T18:57:35.563 INFO:teuthology.orchestra.run.smithi072.stderr:done 2024-03-11T18:57:35.565 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-03-11T18:57:35.580 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.w3rDDkLs72 2024-03-11T18:57:35.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.w3rDDkLs72 2024-03-11T18:57:35.648 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.w3rDDkLs72 2024-03-11T18:57:35.794 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.w3rDDkLs72 is 6KB 2024-03-11T18:57:35.843 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.w3rDDkLs72 2024-03-11T18:57:35.857 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2024-03-11T18:57:35.928 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-11T18:57:35.928 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-11T18:57:35.928 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-11T18:57:35.928 INFO:teuthology.orchestra.run.smithi072.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-11T18:57:35.929 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-11T18:57:36.078 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2024-03-11T18:58:06.081 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-11T18:58:06.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T18:58:06.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:58:24.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-03-11T18:58:33.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T18:58:33.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:58:36.482 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-03-11T18:58:48.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T18:58:48.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:59:48.487 DEBUG:teuthology.orchestra.remote:timed out 2024-03-11T19:00:03.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T19:00:03.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:00:03.950 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-03-11T19:00:04.606 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T19:00:04.606 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-11T19:00:04.606 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-03-11T19:00:04.655 INFO:teuthology.orchestra.run.smithi072.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:00:04.655 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:00:04.655 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-11T19:00:04.655 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-11T19:00:05.656 DEBUG:teuthology.parallel:result is None 2024-03-11T19:00:05.657 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-11T19:00:05.665 INFO:teuthology.task.internal:Creating test directory... 2024-03-11T19:00:05.665 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:00:05.672 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:00:05.675 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:00:05.720 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-11T19:00:05.727 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-11T19:00:05.743 INFO:teuthology.task.internal:Creating archive directory... 2024-03-11T19:00:05.743 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:00:05.751 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:00:05.753 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:00:05.818 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-11T19:00:05.824 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-11T19:00:05.824 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:00:05.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:00:05.845 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:00:05.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:00:05.861 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:00:05.868 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:00:05.888 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:00:05.927 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:05.941 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:05.951 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:05.960 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:06.027 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:06.097 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:00:06.101 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-11T19:00:06.108 INFO:teuthology.task.internal:Configuring sudo... 2024-03-11T19:00:06.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:00:06.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:00:06.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:00:06.198 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-11T19:00:06.208 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-11T19:00:06.208 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:00:06.213 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:00:06.215 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:00:06.260 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:06.346 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:06.450 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:06.532 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:06.625 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:00:06.625 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:00:06.711 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:06.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:06.820 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:06.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:06.909 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:00:06.910 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:00:06.978 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:07.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:00:07.082 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:07.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:00:07.170 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:00:07.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:00:07.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-03-11T19:00:07.245 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-03-11T19:00:07.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-03-11T19:00:07.287 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:00:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:00:07.364 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:00:07.751 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-11T19:00:07.759 INFO:teuthology.task.internal:Starting timer... 2024-03-11T19:00:07.760 INFO:teuthology.run_tasks:Running task pcp... 2024-03-11T19:00:07.805 INFO:teuthology.run_tasks:Running task selinux... 2024-03-11T19:00:07.819 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-11T19:00:07.820 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-03-11T19:00:07.944 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-03-11T19:00:07.948 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-03-11T19:00:08.026 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-03-11T19:00:08.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-03-11T19:00:08.095 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-03-11T19:00:08.097 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-11T19:00:08.097 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-03-11T19:00:08.152 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-03-11T19:00:08.152 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-03-11T19:00:08.175 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-03-11T19:00:08.176 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-03-11T19:00:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-03-11T19:00:08.198 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2024-03-11T19:00:08.198 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:00:08.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:00:08.286 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:00:08.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:00:08.315 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:00:08.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:00:08.343 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-11T19:00:08.343 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:00:08.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:00:08.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:00:08.516 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-11T19:00:08.526 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-11T19:00:08.543 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-11T19:00:08.544 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-11T19:06:42.390 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2024-03-11T19:06:42.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T19:06:42.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:06:42.537 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-03-11T19:06:42.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-03-11T19:06:42.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-11T19:06:42.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:06:42.778 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-03-11T19:06:42.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-11T19:06:42.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-03-11T19:06:42.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:06:42.941 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-03-11T19:06:43.011 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-03-11T19:06:43.011 INFO:teuthology.run_tasks:Running task clock... 2024-03-11T19:06:43.027 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-11T19:06:43.027 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:06:43.028 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:06:43.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:06:43.035 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:06:43.037 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:06:43.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:06:43.070 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:06:43.084 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:06:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:06:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:06:43.117 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-03-11T19:06:43.131 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-03-11T19:06:43.131 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-03-11T19:06:43.132 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:06:43.145 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-03-11T19:06:43.147 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:06:43.164 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:06:43.178 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:06:43.182 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:06:43.207 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.211 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.222 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:06:43.225 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:06:43.225 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-03-11T19:06:43.225 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.225 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.226 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.226 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.268 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-03-11T19:06:43.307 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-03-11T19:06:43.359 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:06:43.411 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:06:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.551 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:06:43.552 INFO:teuthology.run_tasks:Running task install... 2024-03-11T19:06:43.561 DEBUG:teuthology.task.install:project ceph 2024-03-11T19:06:43.561 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-11T19:06:43.561 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-11T19:06:43.561 INFO:teuthology.task.install:Using flavor: default 2024-03-11T19:06:43.567 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-11T19:06:43.567 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-11T19:06:43.567 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:06:43.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:06:43.570 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:06:43.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:06:43.571 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:06:43.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:06:43.716 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:06:43.716 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:06:43.722 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:06:43.723 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:06:43.733 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:06:43.733 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:06:43.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:06:43.825 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:06:43.825 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:06:43.835 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:06:43.835 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:06:43.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:06:43.863 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:06:43.864 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:06:43.902 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:06:43.902 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:06:43.902 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:06:43.922 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:06:43.923 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:06:43.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:06:43.932 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:06:43.932 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:06:44.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:06:44.010 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:06:44.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:06:44.076 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-03-11T19:06:44.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-03-11T19:06:44.081 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:06:44.148 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-03-11T19:06:44.150 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-03-11T19:06:44.177 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:06:44.263 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-03-11T19:06:44.267 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-03-11T19:06:44.312 INFO:teuthology.orchestra.run.smithi143.stdout:74 files removed 2024-03-11T19:06:44.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:06:44.384 INFO:teuthology.orchestra.run.smithi106.stdout:74 files removed 2024-03-11T19:06:44.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:06:44.618 INFO:teuthology.orchestra.run.smithi072.stdout:74 files removed 2024-03-11T19:06:44.656 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:06:45.063 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 189 kB/s | 82 kB 00:00 2024-03-11T19:06:45.190 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 167 kB/s | 82 kB 00:00 2024-03-11T19:06:45.446 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-11T19:06:45.571 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 203 kB/s | 82 kB 00:00 2024-03-11T19:06:45.635 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-11T19:06:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-03-11T19:06:46.050 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-11T19:06:46.172 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 20 kB/s | 10 kB 00:00 2024-03-11T19:06:46.408 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-11T19:06:46.594 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-11T19:06:46.615 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-11T19:06:47.304 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-11T19:06:47.791 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-11T19:06:48.078 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-11T19:06:48.964 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-11T19:06:49.102 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-11T19:06:49.665 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-03-11T19:06:50.434 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-03-11T19:06:52.128 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 19 MB 00:01 2024-03-11T19:06:53.741 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:02 2024-03-11T19:06:54.447 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-03-11T19:06:56.020 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-11T19:06:57.396 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 15 kB 00:00 2024-03-11T19:06:58.539 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-11T19:06:58.700 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.1 MB 00:01 2024-03-11T19:06:59.109 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.1 MB 00:00 2024-03-11T19:07:00.400 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-03-11T19:07:00.803 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 15 kB 00:00 2024-03-11T19:07:01.788 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-11T19:07:02.278 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-11T19:07:03.439 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-11T19:07:05.896 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:05.896 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:05.912 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:07:05.913 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:07:05.914 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:07:05.915 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:05.917 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:05.920 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:07:05.921 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:07:06.001 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-03-11T19:07:06.021 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-11T19:07:06.021 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-03-11T19:07:06.021 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-11T19:07:06.021 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:07:06.022 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:07:06.023 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:06.024 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:07:06.025 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:06.026 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:07:06.027 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:07:06.028 INFO:teuthology.orchestra.run.smithi106.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:07:06.029 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:07:06.030 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:07:06.031 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:07:06.032 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:07:06.033 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:07:06.034 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:07:06.035 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout:Install 150 Packages 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 24 Packages 2024-03-11T19:07:06.036 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:07:06.037 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 223 M 2024-03-11T19:07:06.037 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-03-11T19:07:06.727 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-11T19:07:07.449 INFO:teuthology.orchestra.run.smithi106.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 10 kB/s | 6.6 kB 00:00 2024-03-11T19:07:07.841 INFO:teuthology.orchestra.run.smithi106.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 2.2 MB/s | 886 kB 00:00 2024-03-11T19:07:07.899 INFO:teuthology.orchestra.run.smithi106.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.4 MB/s | 145 kB 00:00 2024-03-11T19:07:08.066 INFO:teuthology.orchestra.run.smithi106.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 13 MB/s | 2.1 MB 00:00 2024-03-11T19:07:08.092 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-11T19:07:08.191 INFO:teuthology.orchestra.run.smithi106.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 12 MB/s | 1.5 MB 00:00 2024-03-11T19:07:08.467 INFO:teuthology.orchestra.run.smithi106.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-11T19:07:09.182 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:09.183 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:09.197 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:07:09.198 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:07:09.199 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:07:09.200 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:09.203 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:09.205 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:07:09.206 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:07:09.286 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:07:09.306 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:07:09.307 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:07:09.308 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:07:09.309 INFO:teuthology.orchestra.run.smithi143.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:07:09.310 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:07:09.311 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:07:09.312 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:07:09.313 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:07:09.314 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:07:09.315 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:07:09.316 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:07:09.317 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:07:09.318 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout:Install 150 Packages 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 24 Packages 2024-03-11T19:07:09.319 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:07:09.320 INFO:teuthology.orchestra.run.smithi106.stdout:(7/174): ceph-osd-19.0.0-1607.g94a53970.el9.x86 20 MB/s | 17 MB 00:00 2024-03-11T19:07:09.320 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 223 M 2024-03-11T19:07:09.320 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-03-11T19:07:09.434 INFO:teuthology.orchestra.run.smithi106.stdout:(8/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 2.0 MB/s | 5.3 MB 00:02 2024-03-11T19:07:09.501 INFO:teuthology.orchestra.run.smithi106.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 378 kB/s | 25 kB 00:00 2024-03-11T19:07:09.809 INFO:teuthology.orchestra.run.smithi106.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 20 MB/s | 9.8 MB 00:00 2024-03-11T19:07:09.901 INFO:teuthology.orchestra.run.smithi106.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 2.3 MB/s | 217 kB 00:00 2024-03-11T19:07:09.968 INFO:teuthology.orchestra.run.smithi106.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 406 kB/s | 27 kB 00:00 2024-03-11T19:07:10.068 INFO:teuthology.orchestra.run.smithi106.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 312 kB/s | 31 kB 00:00 2024-03-11T19:07:10.177 INFO:teuthology.orchestra.run.smithi106.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 6.7 MB/s | 742 kB 00:00 2024-03-11T19:07:10.269 INFO:teuthology.orchestra.run.smithi106.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.7 MB/s | 163 kB 00:00 2024-03-11T19:07:10.352 INFO:teuthology.orchestra.run.smithi106.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.5 MB/s | 127 kB 00:00 2024-03-11T19:07:10.378 INFO:teuthology.orchestra.run.smithi143.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-11T19:07:10.445 INFO:teuthology.orchestra.run.smithi106.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 5.3 MB/s | 498 kB 00:00 2024-03-11T19:07:10.837 INFO:teuthology.orchestra.run.smithi106.stdout:(18/174): librgw2-19.0.0-1607.g94a53970.el9.x86 13 MB/s | 5.1 MB 00:00 2024-03-11T19:07:10.841 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:10.842 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:07:10.863 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:07:10.864 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:07:10.866 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:07:10.867 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:10.869 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:07:10.873 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:07:10.874 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:07:10.895 INFO:teuthology.orchestra.run.smithi106.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 777 kB/s | 45 kB 00:00 2024-03-11T19:07:10.963 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-03-11T19:07:10.970 INFO:teuthology.orchestra.run.smithi106.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.7 MB/s | 131 kB 00:00 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================ 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================ 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:07:10.996 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:07:10.997 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:07:10.998 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:07:10.999 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:07:11.000 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:07:11.001 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:07:11.002 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:07:11.003 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:07:11.004 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:07:11.005 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:07:11.006 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:07:11.007 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-03-11T19:07:11.008 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================================ 2024-03-11T19:07:11.009 INFO:teuthology.orchestra.run.smithi072.stdout:Install 150 Packages 2024-03-11T19:07:11.009 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 24 Packages 2024-03-11T19:07:11.009 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:07:11.009 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 223 M 2024-03-11T19:07:11.009 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-03-11T19:07:11.037 INFO:teuthology.orchestra.run.smithi106.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 2.4 MB/s | 161 kB 00:00 2024-03-11T19:07:11.111 INFO:teuthology.orchestra.run.smithi143.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 1.2 MB/s | 886 kB 00:00 2024-03-11T19:07:11.112 INFO:teuthology.orchestra.run.smithi106.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 4.2 MB/s | 320 kB 00:00 2024-03-11T19:07:11.213 INFO:teuthology.orchestra.run.smithi106.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.9 MB/s | 298 kB 00:00 2024-03-11T19:07:11.228 INFO:teuthology.orchestra.run.smithi143.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.2 MB/s | 145 kB 00:00 2024-03-11T19:07:11.280 INFO:teuthology.orchestra.run.smithi106.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.5 MB/s | 99 kB 00:00 2024-03-11T19:07:11.347 INFO:teuthology.orchestra.run.smithi106.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-11T19:07:11.647 INFO:teuthology.orchestra.run.smithi106.stdout:(26/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 10 MB/s | 3.1 MB 00:00 2024-03-11T19:07:11.703 INFO:teuthology.orchestra.run.smithi143.stdout:(4/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 3.4 MB/s | 5.3 MB 00:01 2024-03-11T19:07:11.714 INFO:teuthology.orchestra.run.smithi106.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-11T19:07:11.781 INFO:teuthology.orchestra.run.smithi106.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 471 kB/s | 31 kB 00:00 2024-03-11T19:07:11.856 INFO:teuthology.orchestra.run.smithi106.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.8 MB/s | 138 kB 00:00 2024-03-11T19:07:11.904 INFO:teuthology.orchestra.run.smithi143.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 7.4 MB/s | 1.5 MB 00:00 2024-03-11T19:07:11.912 INFO:teuthology.orchestra.run.smithi072.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-11T19:07:12.031 INFO:teuthology.orchestra.run.smithi106.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 10 MB/s | 1.8 MB 00:00 2024-03-11T19:07:12.715 INFO:teuthology.orchestra.run.smithi106.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-11T19:07:12.745 INFO:teuthology.orchestra.run.smithi143.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 5.6 MB/s | 4.7 MB 00:00 2024-03-11T19:07:12.815 INFO:teuthology.orchestra.run.smithi106.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.4 MB/s | 247 kB 00:00 2024-03-11T19:07:12.821 INFO:teuthology.orchestra.run.smithi143.stdout:(7/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 1.3 MB/s | 2.1 MB 00:01 2024-03-11T19:07:12.902 INFO:teuthology.orchestra.run.smithi106.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 573 kB/s | 49 kB 00:00 2024-03-11T19:07:12.966 INFO:teuthology.orchestra.run.smithi106.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 239 kB/s | 15 kB 00:00 2024-03-11T19:07:13.050 INFO:teuthology.orchestra.run.smithi106.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 3.1 MB/s | 264 kB 00:00 2024-03-11T19:07:13.087 INFO:teuthology.orchestra.run.smithi072.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 754 kB/s | 886 kB 00:01 2024-03-11T19:07:13.158 INFO:teuthology.orchestra.run.smithi106.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 6.8 MB/s | 748 kB 00:00 2024-03-11T19:07:13.225 INFO:teuthology.orchestra.run.smithi106.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 274 kB/s | 18 kB 00:00 2024-03-11T19:07:13.263 INFO:teuthology.orchestra.run.smithi072.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 835 kB/s | 145 kB 00:00 2024-03-11T19:07:13.359 INFO:teuthology.orchestra.run.smithi106.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-11T19:07:13.417 INFO:teuthology.orchestra.run.smithi106.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-11T19:07:13.443 INFO:teuthology.orchestra.run.smithi106.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 4.1 MB/s | 106 kB 00:00 2024-03-11T19:07:13.485 INFO:teuthology.orchestra.run.smithi106.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-11T19:07:13.526 INFO:teuthology.orchestra.run.smithi106.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 762 kB/s | 32 kB 00:00 2024-03-11T19:07:13.643 INFO:teuthology.orchestra.run.smithi106.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-11T19:07:13.685 INFO:teuthology.orchestra.run.smithi106.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 412 kB/s | 17 kB 00:00 2024-03-11T19:07:13.752 INFO:teuthology.orchestra.run.smithi106.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 151 kB/s | 10 kB 00:00 2024-03-11T19:07:13.828 INFO:teuthology.orchestra.run.smithi106.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-11T19:07:13.870 INFO:teuthology.orchestra.run.smithi106.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-11T19:07:13.928 INFO:teuthology.orchestra.run.smithi106.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-11T19:07:13.995 INFO:teuthology.orchestra.run.smithi106.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-11T19:07:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 796 kB/s | 46 kB 00:00 2024-03-11T19:07:14.096 INFO:teuthology.orchestra.run.smithi106.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-11T19:07:14.179 INFO:teuthology.orchestra.run.smithi106.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-11T19:07:14.221 INFO:teuthology.orchestra.run.smithi106.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-11T19:07:14.296 INFO:teuthology.orchestra.run.smithi106.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-03-11T19:07:14.468 INFO:teuthology.orchestra.run.smithi106.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-03-11T19:07:14.535 INFO:teuthology.orchestra.run.smithi106.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-03-11T19:07:14.594 INFO:teuthology.orchestra.run.smithi106.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-11T19:07:14.612 INFO:teuthology.orchestra.run.smithi143.stdout:(8/174): ceph-osd-19.0.0-1607.g94a53970.el9.x86 9.0 MB/s | 17 MB 00:01 2024-03-11T19:07:14.636 INFO:teuthology.orchestra.run.smithi106.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 3.1 MB/s | 134 kB 00:00 2024-03-11T19:07:14.671 INFO:teuthology.orchestra.run.smithi143.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 434 kB/s | 25 kB 00:00 2024-03-11T19:07:14.820 INFO:teuthology.orchestra.run.smithi106.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-11T19:07:14.830 INFO:teuthology.orchestra.run.smithi072.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 1.4 MB/s | 2.1 MB 00:01 2024-03-11T19:07:14.887 INFO:teuthology.orchestra.run.smithi106.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-11T19:07:14.937 INFO:teuthology.orchestra.run.smithi106.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 78 kB 00:00 2024-03-11T19:07:15.112 INFO:teuthology.orchestra.run.smithi106.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-11T19:07:15.154 INFO:teuthology.orchestra.run.smithi106.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 590 kB/s | 24 kB 00:00 2024-03-11T19:07:15.196 INFO:teuthology.orchestra.run.smithi106.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 436 kB/s | 18 kB 00:00 2024-03-11T19:07:15.238 INFO:teuthology.orchestra.run.smithi106.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-11T19:07:15.280 INFO:teuthology.orchestra.run.smithi106.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 758 kB/s | 32 kB 00:00 2024-03-11T19:07:15.439 INFO:teuthology.orchestra.run.smithi106.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-11T19:07:15.481 INFO:teuthology.orchestra.run.smithi106.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2024-03-11T19:07:15.531 INFO:teuthology.orchestra.run.smithi106.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-11T19:07:15.581 INFO:teuthology.orchestra.run.smithi106.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-11T19:07:15.706 INFO:teuthology.orchestra.run.smithi106.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 329 kB/s | 41 kB 00:00 2024-03-11T19:07:15.714 INFO:teuthology.orchestra.run.smithi072.stdout:(5/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 1.3 MB/s | 5.3 MB 00:04 2024-03-11T19:07:15.807 INFO:teuthology.orchestra.run.smithi106.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 238 kB 00:00 2024-03-11T19:07:15.829 INFO:teuthology.orchestra.run.smithi143.stdout:(10/174): ceph-common-19.0.0-1607.g94a53970.el9 3.5 MB/s | 20 MB 00:05 2024-03-11T19:07:15.898 INFO:teuthology.orchestra.run.smithi072.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 1.4 MB/s | 1.5 MB 00:01 2024-03-11T19:07:15.915 INFO:teuthology.orchestra.run.smithi106.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-03-11T19:07:15.963 INFO:teuthology.orchestra.run.smithi143.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.6 MB/s | 217 kB 00:00 2024-03-11T19:07:15.965 INFO:teuthology.orchestra.run.smithi106.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-11T19:07:16.030 INFO:teuthology.orchestra.run.smithi143.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 406 kB/s | 27 kB 00:00 2024-03-11T19:07:16.032 INFO:teuthology.orchestra.run.smithi106.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 712 kB/s | 47 kB 00:00 2024-03-11T19:07:16.066 INFO:teuthology.orchestra.run.smithi106.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-11T19:07:16.097 INFO:teuthology.orchestra.run.smithi143.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 468 kB/s | 31 kB 00:00 2024-03-11T19:07:16.364 INFO:teuthology.orchestra.run.smithi143.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 2.7 MB/s | 742 kB 00:00 2024-03-11T19:07:16.439 INFO:teuthology.orchestra.run.smithi143.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 2.1 MB/s | 163 kB 00:00 2024-03-11T19:07:16.514 INFO:teuthology.orchestra.run.smithi143.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.6 MB/s | 127 kB 00:00 2024-03-11T19:07:16.642 INFO:teuthology.orchestra.run.smithi106.stdout:(77/174): ceph-test-19.0.0-1607.g94a53970.el9.x 6.6 MB/s | 47 MB 00:07 2024-03-11T19:07:16.681 INFO:teuthology.orchestra.run.smithi143.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 2.9 MB/s | 498 kB 00:00 2024-03-11T19:07:16.925 INFO:teuthology.orchestra.run.smithi106.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-03-11T19:07:16.951 INFO:teuthology.orchestra.run.smithi106.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 126 kB/s | 39 kB 00:00 2024-03-11T19:07:16.976 INFO:teuthology.orchestra.run.smithi106.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 822 kB/s | 41 kB 00:00 2024-03-11T19:07:17.009 INFO:teuthology.orchestra.run.smithi106.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-11T19:07:17.034 INFO:teuthology.orchestra.run.smithi106.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 968 kB/s | 56 kB 00:00 2024-03-11T19:07:17.060 INFO:teuthology.orchestra.run.smithi106.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-11T19:07:17.085 INFO:teuthology.orchestra.run.smithi106.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-03-11T19:07:17.119 INFO:teuthology.orchestra.run.smithi106.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-11T19:07:17.144 INFO:teuthology.orchestra.run.smithi106.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-11T19:07:17.195 INFO:teuthology.orchestra.run.smithi106.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-11T19:07:17.220 INFO:teuthology.orchestra.run.smithi106.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-11T19:07:17.245 INFO:teuthology.orchestra.run.smithi106.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-11T19:07:17.270 INFO:teuthology.orchestra.run.smithi106.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-11T19:07:17.420 INFO:teuthology.orchestra.run.smithi106.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-11T19:07:17.513 INFO:teuthology.orchestra.run.smithi106.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-03-11T19:07:17.538 INFO:teuthology.orchestra.run.smithi106.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-03-11T19:07:17.563 INFO:teuthology.orchestra.run.smithi106.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-11T19:07:17.740 INFO:teuthology.orchestra.run.smithi143.stdout:(18/174): ceph-radosgw-19.0.0-1607.g94a53970.el 2.0 MB/s | 9.8 MB 00:04 2024-03-11T19:07:17.824 INFO:teuthology.orchestra.run.smithi143.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 541 kB/s | 45 kB 00:00 2024-03-11T19:07:17.899 INFO:teuthology.orchestra.run.smithi143.stdout:(20/174): librgw2-19.0.0-1607.g94a53970.el9.x86 4.2 MB/s | 5.1 MB 00:01 2024-03-11T19:07:17.908 INFO:teuthology.orchestra.run.smithi072.stdout:(7/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-03-11T19:07:17.949 INFO:teuthology.orchestra.run.smithi143.stdout:(21/174): python3-ceph-common-19.0.0-1607.g94a5 1.0 MB/s | 131 kB 00:00 2024-03-11T19:07:17.974 INFO:teuthology.orchestra.run.smithi143.stdout:(22/174): python3-cephfs-19.0.0-1607.g94a53970. 2.1 MB/s | 161 kB 00:00 2024-03-11T19:07:18.091 INFO:teuthology.orchestra.run.smithi143.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.5 MB/s | 298 kB 00:00 2024-03-11T19:07:18.117 INFO:teuthology.orchestra.run.smithi143.stdout:(24/174): python3-rados-19.0.0-1607.g94a53970.e 1.9 MB/s | 320 kB 00:00 2024-03-11T19:07:18.167 INFO:teuthology.orchestra.run.smithi143.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.3 MB/s | 99 kB 00:00 2024-03-11T19:07:18.188 INFO:teuthology.orchestra.run.smithi106.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 33 kB/s | 21 kB 00:00 2024-03-11T19:07:18.192 INFO:teuthology.orchestra.run.smithi143.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-11T19:07:18.222 INFO:teuthology.orchestra.run.smithi106.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 39 kB/s | 25 kB 00:00 2024-03-11T19:07:18.317 INFO:teuthology.orchestra.run.smithi143.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.3 MB/s | 171 kB 00:00 2024-03-11T19:07:18.392 INFO:teuthology.orchestra.run.smithi143.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 420 kB/s | 31 kB 00:00 2024-03-11T19:07:18.469 INFO:teuthology.orchestra.run.smithi143.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.8 MB/s | 138 kB 00:00 2024-03-11T19:07:18.885 INFO:teuthology.orchestra.run.smithi143.stdout:(30/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 4.3 MB/s | 3.1 MB 00:00 2024-03-11T19:07:19.222 INFO:teuthology.orchestra.run.smithi106.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 245 kB/s | 245 kB 00:00 2024-03-11T19:07:19.493 INFO:teuthology.orchestra.run.smithi143.stdout:(31/174): ceph-test-19.0.0-1607.g94a53970.el9.x 9.8 MB/s | 47 MB 00:04 2024-03-11T19:07:19.536 INFO:teuthology.orchestra.run.smithi143.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 1.7 MB/s | 1.8 MB 00:01 2024-03-11T19:07:19.611 INFO:teuthology.orchestra.run.smithi143.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.1 MB/s | 247 kB 00:00 2024-03-11T19:07:19.636 INFO:teuthology.orchestra.run.smithi143.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 494 kB/s | 49 kB 00:00 2024-03-11T19:07:19.686 INFO:teuthology.orchestra.run.smithi143.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 204 kB/s | 15 kB 00:00 2024-03-11T19:07:19.803 INFO:teuthology.orchestra.run.smithi143.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.5 MB/s | 264 kB 00:00 2024-03-11T19:07:19.845 INFO:teuthology.orchestra.run.smithi143.stdout:(37/174): cephadm-19.0.0-1607.g94a53970.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-11T19:07:19.870 INFO:teuthology.orchestra.run.smithi143.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 273 kB/s | 18 kB 00:00 2024-03-11T19:07:19.937 INFO:teuthology.orchestra.run.smithi072.stdout:(8/174): ceph-common-19.0.0-1607.g94a53970.el9. 2.4 MB/s | 20 MB 00:08 2024-03-11T19:07:20.059 INFO:teuthology.orchestra.run.smithi072.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 208 kB/s | 25 kB 00:00 2024-03-11T19:07:20.070 INFO:teuthology.orchestra.run.smithi143.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 662 kB 00:00 2024-03-11T19:07:20.113 INFO:teuthology.orchestra.run.smithi143.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-03-11T19:07:20.138 INFO:teuthology.orchestra.run.smithi143.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-11T19:07:20.163 INFO:teuthology.orchestra.run.smithi143.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-11T19:07:20.186 INFO:teuthology.orchestra.run.smithi143.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 669 kB/s | 32 kB 00:00 2024-03-11T19:07:20.277 INFO:teuthology.orchestra.run.smithi143.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-11T19:07:20.280 INFO:teuthology.orchestra.run.smithi106.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 234 kB/s | 248 kB 00:01 2024-03-11T19:07:20.302 INFO:teuthology.orchestra.run.smithi143.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-11T19:07:20.328 INFO:teuthology.orchestra.run.smithi143.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-11T19:07:20.353 INFO:teuthology.orchestra.run.smithi143.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-11T19:07:20.379 INFO:teuthology.orchestra.run.smithi143.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-11T19:07:20.404 INFO:teuthology.orchestra.run.smithi143.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-11T19:07:20.454 INFO:teuthology.orchestra.run.smithi143.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-11T19:07:20.480 INFO:teuthology.orchestra.run.smithi143.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 615 kB/s | 46 kB 00:00 2024-03-11T19:07:20.530 INFO:teuthology.orchestra.run.smithi143.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-11T19:07:20.555 INFO:teuthology.orchestra.run.smithi143.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-11T19:07:20.597 INFO:teuthology.orchestra.run.smithi143.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-11T19:07:20.614 INFO:teuthology.orchestra.run.smithi143.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 582 kB/s | 34 kB 00:00 2024-03-11T19:07:20.756 INFO:teuthology.orchestra.run.smithi143.stdout:(56/174): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-03-11T19:07:20.832 INFO:teuthology.orchestra.run.smithi143.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-11T19:07:20.865 INFO:teuthology.orchestra.run.smithi143.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2024-03-11T19:07:20.898 INFO:teuthology.orchestra.run.smithi143.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.7 MB/s | 811 kB 00:00 2024-03-11T19:07:20.923 INFO:teuthology.orchestra.run.smithi143.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-11T19:07:20.991 INFO:teuthology.orchestra.run.smithi143.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-11T19:07:21.099 INFO:teuthology.orchestra.run.smithi143.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-03-11T19:07:21.124 INFO:teuthology.orchestra.run.smithi143.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 387 kB/s | 78 kB 00:00 2024-03-11T19:07:21.225 INFO:teuthology.orchestra.run.smithi143.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-11T19:07:21.250 INFO:teuthology.orchestra.run.smithi143.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-11T19:07:21.275 INFO:teuthology.orchestra.run.smithi143.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-03-11T19:07:21.317 INFO:teuthology.orchestra.run.smithi143.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 766 kB/s | 32 kB 00:00 2024-03-11T19:07:21.342 INFO:teuthology.orchestra.run.smithi143.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-11T19:07:21.389 INFO:teuthology.orchestra.run.smithi106.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 195 kB/s | 216 kB 00:01 2024-03-11T19:07:21.434 INFO:teuthology.orchestra.run.smithi143.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-11T19:07:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 2.8 MB/s | 9.8 MB 00:03 2024-03-11T19:07:21.460 INFO:teuthology.orchestra.run.smithi143.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-11T19:07:21.493 INFO:teuthology.orchestra.run.smithi143.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-11T19:07:21.519 INFO:teuthology.orchestra.run.smithi143.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 827 kB/s | 69 kB 00:00 2024-03-11T19:07:21.544 INFO:teuthology.orchestra.run.smithi143.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-11T19:07:21.569 INFO:teuthology.orchestra.run.smithi143.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-11T19:07:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.5 MB/s | 217 kB 00:00 2024-03-11T19:07:21.594 INFO:teuthology.orchestra.run.smithi143.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-11T19:07:21.628 INFO:teuthology.orchestra.run.smithi143.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-03-11T19:07:21.648 INFO:teuthology.orchestra.run.smithi106.stdout:(100/174): ceph-common-19.0.0-1607.g94a53970.el 1.3 MB/s | 20 MB 00:14 2024-03-11T19:07:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 410 kB/s | 27 kB 00:00 2024-03-11T19:07:21.653 INFO:teuthology.orchestra.run.smithi143.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 570 kB/s | 47 kB 00:00 2024-03-11T19:07:21.678 INFO:teuthology.orchestra.run.smithi143.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 512 kB/s | 43 kB 00:00 2024-03-11T19:07:21.703 INFO:teuthology.orchestra.run.smithi143.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 774 kB/s | 39 kB 00:00 2024-03-11T19:07:21.719 INFO:teuthology.orchestra.run.smithi072.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 471 kB/s | 31 kB 00:00 2024-03-11T19:07:21.804 INFO:teuthology.orchestra.run.smithi143.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 328 kB/s | 41 kB 00:00 2024-03-11T19:07:21.921 INFO:teuthology.orchestra.run.smithi143.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-11T19:07:22.012 INFO:teuthology.orchestra.run.smithi072.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 2.5 MB/s | 742 kB 00:00 2024-03-11T19:07:22.079 INFO:teuthology.orchestra.run.smithi143.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-11T19:07:22.104 INFO:teuthology.orchestra.run.smithi143.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 189 kB/s | 56 kB 00:00 2024-03-11T19:07:22.129 INFO:teuthology.orchestra.run.smithi072.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.4 MB/s | 163 kB 00:00 2024-03-11T19:07:22.130 INFO:teuthology.orchestra.run.smithi143.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 301 kB/s | 63 kB 00:00 2024-03-11T19:07:22.155 INFO:teuthology.orchestra.run.smithi143.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-11T19:07:22.180 INFO:teuthology.orchestra.run.smithi143.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-11T19:07:22.205 INFO:teuthology.orchestra.run.smithi072.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.7 MB/s | 127 kB 00:00 2024-03-11T19:07:22.205 INFO:teuthology.orchestra.run.smithi143.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-11T19:07:22.247 INFO:teuthology.orchestra.run.smithi143.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-11T19:07:22.289 INFO:teuthology.orchestra.run.smithi143.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-11T19:07:22.290 INFO:teuthology.orchestra.run.smithi106.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 39 kB/s | 35 kB 00:00 2024-03-11T19:07:22.314 INFO:teuthology.orchestra.run.smithi143.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-11T19:07:22.339 INFO:teuthology.orchestra.run.smithi143.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-11T19:07:22.390 INFO:teuthology.orchestra.run.smithi106.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 60 kB/s | 44 kB 00:00 2024-03-11T19:07:22.665 INFO:teuthology.orchestra.run.smithi106.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-03-11T19:07:22.697 INFO:teuthology.orchestra.run.smithi072.stdout:(17/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 2.5 MB/s | 17 MB 00:06 2024-03-11T19:07:22.747 INFO:teuthology.orchestra.run.smithi072.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 919 kB/s | 498 kB 00:00 2024-03-11T19:07:22.790 INFO:teuthology.orchestra.run.smithi143.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 394 kB/s | 187 kB 00:00 2024-03-11T19:07:22.840 INFO:teuthology.orchestra.run.smithi072.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 496 kB/s | 45 kB 00:00 2024-03-11T19:07:22.890 INFO:teuthology.orchestra.run.smithi143.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 355 kB/s | 195 kB 00:00 2024-03-11T19:07:22.915 INFO:teuthology.orchestra.run.smithi143.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 998 kB/s | 125 kB 00:00 2024-03-11T19:07:22.982 INFO:teuthology.orchestra.run.smithi072.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 930 kB/s | 131 kB 00:00 2024-03-11T19:07:23.032 INFO:teuthology.orchestra.run.smithi143.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-03-11T19:07:23.057 INFO:teuthology.orchestra.run.smithi143.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-11T19:07:23.098 INFO:teuthology.orchestra.run.smithi143.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-11T19:07:23.134 INFO:teuthology.orchestra.run.smithi072.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.0 MB/s | 161 kB 00:00 2024-03-11T19:07:23.174 INFO:teuthology.orchestra.run.smithi106.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 106 kB/s | 94 kB 00:00 2024-03-11T19:07:23.198 INFO:teuthology.orchestra.run.smithi143.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-03-11T19:07:23.248 INFO:teuthology.orchestra.run.smithi143.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-11T19:07:23.316 INFO:teuthology.orchestra.run.smithi106.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-03-11T19:07:23.340 INFO:teuthology.orchestra.run.smithi143.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.5 MB/s | 216 kB 00:00 2024-03-11T19:07:23.376 INFO:teuthology.orchestra.run.smithi072.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 1.3 MB/s | 320 kB 00:00 2024-03-11T19:07:23.382 INFO:teuthology.orchestra.run.smithi143.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 261 kB/s | 35 kB 00:00 2024-03-11T19:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 355 kB/s | 44 kB 00:00 2024-03-11T19:07:23.499 INFO:teuthology.orchestra.run.smithi143.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 803 kB/s | 94 kB 00:00 2024-03-11T19:07:23.518 INFO:teuthology.orchestra.run.smithi072.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.1 MB/s | 298 kB 00:00 2024-03-11T19:07:23.558 INFO:teuthology.orchestra.run.smithi143.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-03-11T19:07:23.594 INFO:teuthology.orchestra.run.smithi072.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.3 MB/s | 99 kB 00:00 2024-03-11T19:07:23.658 INFO:teuthology.orchestra.run.smithi143.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-03-11T19:07:23.670 INFO:teuthology.orchestra.run.smithi072.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-11T19:07:23.684 INFO:teuthology.orchestra.run.smithi106.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-11T19:07:23.709 INFO:teuthology.orchestra.run.smithi106.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 786 kB/s | 309 kB 00:00 2024-03-11T19:07:23.826 INFO:teuthology.orchestra.run.smithi106.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-03-11T19:07:23.909 INFO:teuthology.orchestra.run.smithi106.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-03-11T19:07:24.002 INFO:teuthology.orchestra.run.smithi106.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-03-11T19:07:24.102 INFO:teuthology.orchestra.run.smithi106.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-03-11T19:07:24.211 INFO:teuthology.orchestra.run.smithi106.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-11T19:07:24.302 INFO:teuthology.orchestra.run.smithi106.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-03-11T19:07:24.403 INFO:teuthology.orchestra.run.smithi106.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-03-11T19:07:24.447 INFO:teuthology.orchestra.run.smithi106.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-03-11T19:07:24.566 INFO:teuthology.orchestra.run.smithi106.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 266 kB/s | 43 kB 00:00 2024-03-11T19:07:24.697 INFO:teuthology.orchestra.run.smithi106.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 115 kB/s | 15 kB 00:00 2024-03-11T19:07:24.814 INFO:teuthology.orchestra.run.smithi106.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-11T19:07:24.931 INFO:teuthology.orchestra.run.smithi106.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 66 kB/s | 32 kB 00:00 2024-03-11T19:07:25.081 INFO:teuthology.orchestra.run.smithi106.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 1.3 MB/s | 358 kB 00:00 2024-03-11T19:07:25.182 INFO:teuthology.orchestra.run.smithi106.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-03-11T19:07:25.290 INFO:teuthology.orchestra.run.smithi106.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-11T19:07:25.445 INFO:teuthology.orchestra.run.smithi072.stdout:(26/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 1.7 MB/s | 3.1 MB 00:01 2024-03-11T19:07:25.540 INFO:teuthology.orchestra.run.smithi106.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.2 MB/s | 2.0 MB 00:00 2024-03-11T19:07:25.588 INFO:teuthology.orchestra.run.smithi072.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.2 MB/s | 171 kB 00:00 2024-03-11T19:07:25.599 INFO:teuthology.orchestra.run.smithi106.stdout:(124/174): openssl-devel-3.0.7-27.el9.x86_64.rp 571 kB/s | 4.1 MB 00:07 2024-03-11T19:07:25.624 INFO:teuthology.orchestra.run.smithi106.stdout:(125/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 437 kB/s | 303 kB 00:00 2024-03-11T19:07:25.649 INFO:teuthology.orchestra.run.smithi106.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-11T19:07:25.655 INFO:teuthology.orchestra.run.smithi072.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 471 kB/s | 31 kB 00:00 2024-03-11T19:07:25.708 INFO:teuthology.orchestra.run.smithi106.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-03-11T19:07:25.733 INFO:teuthology.orchestra.run.smithi106.stdout:(128/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-11T19:07:25.759 INFO:teuthology.orchestra.run.smithi106.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-03-11T19:07:25.792 INFO:teuthology.orchestra.run.smithi106.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-11T19:07:25.805 INFO:teuthology.orchestra.run.smithi072.stdout:(29/174): librgw2-19.0.0-1607.g94a53970.el9.x86 1.6 MB/s | 5.1 MB 00:03 2024-03-11T19:07:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-11T19:07:25.841 INFO:teuthology.orchestra.run.smithi143.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 141 kB/s | 309 kB 00:02 2024-03-11T19:07:25.867 INFO:teuthology.orchestra.run.smithi143.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 237 kB/s | 548 kB 00:02 2024-03-11T19:07:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-03-11T19:07:25.873 INFO:teuthology.orchestra.run.smithi072.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 637 kB/s | 138 kB 00:00 2024-03-11T19:07:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 942 kB/s | 79 kB 00:00 2024-03-11T19:07:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-11T19:07:26.019 INFO:teuthology.orchestra.run.smithi106.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-11T19:07:26.044 INFO:teuthology.orchestra.run.smithi106.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-11T19:07:26.086 INFO:teuthology.orchestra.run.smithi106.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-11T19:07:26.119 INFO:teuthology.orchestra.run.smithi106.stdout:(138/174): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-03-11T19:07:26.144 INFO:teuthology.orchestra.run.smithi106.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-03-11T19:07:26.170 INFO:teuthology.orchestra.run.smithi106.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-03-11T19:07:26.270 INFO:teuthology.orchestra.run.smithi106.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-03-11T19:07:26.287 INFO:teuthology.orchestra.run.smithi106.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-03-11T19:07:26.304 INFO:teuthology.orchestra.run.smithi106.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-03-11T19:07:26.329 INFO:teuthology.orchestra.run.smithi106.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-03-11T19:07:26.388 INFO:teuthology.orchestra.run.smithi106.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-11T19:07:26.413 INFO:teuthology.orchestra.run.smithi106.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-11T19:07:26.442 INFO:teuthology.orchestra.run.smithi143.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 32 kB/s | 19 kB 00:00 2024-03-11T19:07:26.472 INFO:teuthology.orchestra.run.smithi106.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-11T19:07:26.489 INFO:teuthology.orchestra.run.smithi106.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-03-11T19:07:26.589 INFO:teuthology.orchestra.run.smithi106.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-11T19:07:26.740 INFO:teuthology.orchestra.run.smithi072.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 1.9 MB/s | 1.8 MB 00:00 2024-03-11T19:07:26.773 INFO:teuthology.orchestra.run.smithi106.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-03-11T19:07:26.815 INFO:teuthology.orchestra.run.smithi106.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-11T19:07:26.909 INFO:teuthology.orchestra.run.smithi143.stdout:(109/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.1 MB/s | 4.1 MB 00:03 2024-03-11T19:07:26.932 INFO:teuthology.orchestra.run.smithi072.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 1.3 MB/s | 247 kB 00:00 2024-03-11T19:07:26.948 INFO:teuthology.orchestra.run.smithi106.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 7.3 MB/s | 3.4 MB 00:00 2024-03-11T19:07:26.999 INFO:teuthology.orchestra.run.smithi072.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 747 kB/s | 49 kB 00:00 2024-03-11T19:07:27.058 INFO:teuthology.orchestra.run.smithi072.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 265 kB/s | 15 kB 00:00 2024-03-11T19:07:27.101 INFO:teuthology.orchestra.run.smithi143.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-03-11T19:07:27.198 INFO:teuthology.orchestra.run.smithi106.stdout:(153/174): librbd1-19.0.0-1607.g94a53970.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-03-11T19:07:27.201 INFO:teuthology.orchestra.run.smithi143.stdout:(111/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 34 kB/s | 26 kB 00:00 2024-03-11T19:07:27.201 INFO:teuthology.orchestra.run.smithi072.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.8 MB/s | 264 kB 00:00 2024-03-11T19:07:27.315 INFO:teuthology.orchestra.run.smithi106.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 5.0 MB/s | 1.8 MB 00:00 2024-03-11T19:07:27.357 INFO:teuthology.orchestra.run.smithi106.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 267 kB/s | 42 kB 00:00 2024-03-11T19:07:27.407 INFO:teuthology.orchestra.run.smithi106.stdout:(156/174): NetworkManager-1.46.0-1.el9.x86_64.r 3.9 MB/s | 2.3 MB 00:00 2024-03-11T19:07:27.432 INFO:teuthology.orchestra.run.smithi106.stdout:(157/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-03-11T19:07:27.458 INFO:teuthology.orchestra.run.smithi106.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 711 kB/s | 71 kB 00:00 2024-03-11T19:07:27.475 INFO:teuthology.orchestra.run.smithi106.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 542 kB/s | 36 kB 00:00 2024-03-11T19:07:27.500 INFO:teuthology.orchestra.run.smithi106.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-11T19:07:27.517 INFO:teuthology.orchestra.run.smithi106.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 550 kB/s | 23 kB 00:00 2024-03-11T19:07:27.551 INFO:teuthology.orchestra.run.smithi106.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-11T19:07:27.569 INFO:teuthology.orchestra.run.smithi072.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 2.0 MB/s | 748 kB 00:00 2024-03-11T19:07:27.576 INFO:teuthology.orchestra.run.smithi106.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-11T19:07:27.593 INFO:teuthology.orchestra.run.smithi106.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-11T19:07:27.618 INFO:teuthology.orchestra.run.smithi106.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-11T19:07:27.636 INFO:teuthology.orchestra.run.smithi072.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 276 kB/s | 18 kB 00:00 2024-03-11T19:07:27.644 INFO:teuthology.orchestra.run.smithi106.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 951 kB/s | 64 kB 00:00 2024-03-11T19:07:27.669 INFO:teuthology.orchestra.run.smithi106.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-11T19:07:27.786 INFO:teuthology.orchestra.run.smithi106.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 7.2 MB/s | 1.2 MB 00:00 2024-03-11T19:07:27.837 INFO:teuthology.orchestra.run.smithi072.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-11T19:07:27.869 INFO:teuthology.orchestra.run.smithi106.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 5.5 MB/s | 465 kB 00:00 2024-03-11T19:07:27.904 INFO:teuthology.orchestra.run.smithi072.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-03-11T19:07:27.911 INFO:teuthology.orchestra.run.smithi106.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 750 kB/s | 31 kB 00:00 2024-03-11T19:07:27.971 INFO:teuthology.orchestra.run.smithi072.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-11T19:07:28.022 INFO:teuthology.orchestra.run.smithi072.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 307 kB 00:00 2024-03-11T19:07:28.028 INFO:teuthology.orchestra.run.smithi106.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-03-11T19:07:28.073 INFO:teuthology.orchestra.run.smithi072.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2024-03-11T19:07:28.095 INFO:teuthology.orchestra.run.smithi106.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.8 MB/s | 2.2 MB 00:00 2024-03-11T19:07:28.101 INFO:teuthology.orchestra.run.smithi143.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 67 kB/s | 67 kB 00:00 2024-03-11T19:07:28.187 INFO:teuthology.orchestra.run.smithi106.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.4 MB/s | 2.3 MB 00:00 2024-03-11T19:07:28.215 INFO:teuthology.orchestra.run.smithi072.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-11T19:07:28.249 INFO:teuthology.orchestra.run.smithi072.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 523 kB/s | 17 kB 00:00 2024-03-11T19:07:28.282 INFO:teuthology.orchestra.run.smithi072.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 307 kB/s | 10 kB 00:00 2024-03-11T19:07:28.325 INFO:teuthology.orchestra.run.smithi072.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-03-11T19:07:28.359 INFO:teuthology.orchestra.run.smithi072.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-11T19:07:28.418 INFO:teuthology.orchestra.run.smithi072.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-11T19:07:28.451 INFO:teuthology.orchestra.run.smithi143.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 121 kB/s | 151 kB 00:01 2024-03-11T19:07:28.470 INFO:teuthology.orchestra.run.smithi072.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-11T19:07:28.512 INFO:teuthology.orchestra.run.smithi072.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-11T19:07:28.563 INFO:teuthology.orchestra.run.smithi072.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-03-11T19:07:28.613 INFO:teuthology.orchestra.run.smithi072.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-11T19:07:28.656 INFO:teuthology.orchestra.run.smithi072.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2024-03-11T19:07:28.706 INFO:teuthology.orchestra.run.smithi072.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 690 kB/s | 34 kB 00:00 2024-03-11T19:07:28.820 INFO:teuthology.orchestra.run.smithi106.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 94 kB/s | 74 kB 00:00 2024-03-11T19:07:28.824 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:07:28.824 INFO:teuthology.orchestra.run.smithi106.stdout:Total 9.8 MB/s | 223 MB 00:22 2024-03-11T19:07:28.907 INFO:teuthology.orchestra.run.smithi072.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-11T19:07:28.991 INFO:teuthology.orchestra.run.smithi072.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-03-11T19:07:29.058 INFO:teuthology.orchestra.run.smithi072.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-11T19:07:29.109 INFO:teuthology.orchestra.run.smithi072.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 2.6 MB/s | 134 kB 00:00 2024-03-11T19:07:29.303 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-03-11T19:07:29.335 INFO:teuthology.orchestra.run.smithi072.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-11T19:07:29.410 INFO:teuthology.orchestra.run.smithi072.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-03-11T19:07:29.452 INFO:teuthology.orchestra.run.smithi072.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-03-11T19:07:29.576 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-03-11T19:07:29.576 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-03-11T19:07:29.688 INFO:teuthology.orchestra.run.smithi072.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-11T19:07:29.880 INFO:teuthology.orchestra.run.smithi072.stdout:(63/174): ceph-mgr-diskprediction-local-19.0.0- 1.8 MB/s | 7.4 MB 00:04 2024-03-11T19:07:29.906 INFO:teuthology.orchestra.run.smithi072.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 113 kB/s | 24 kB 00:00 2024-03-11T19:07:29.931 INFO:teuthology.orchestra.run.smithi072.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 364 kB/s | 18 kB 00:00 2024-03-11T19:07:29.965 INFO:teuthology.orchestra.run.smithi072.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-11T19:07:29.999 INFO:teuthology.orchestra.run.smithi072.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 483 kB/s | 32 kB 00:00 2024-03-11T19:07:30.467 INFO:teuthology.orchestra.run.smithi072.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.6 MB/s | 4.8 MB 00:00 2024-03-11T19:07:30.525 INFO:teuthology.orchestra.run.smithi072.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 418 kB/s | 217 kB 00:00 2024-03-11T19:07:30.584 INFO:teuthology.orchestra.run.smithi072.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 174 kB 00:00 2024-03-11T19:07:30.618 INFO:teuthology.orchestra.run.smithi072.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 753 kB/s | 69 kB 00:00 2024-03-11T19:07:30.802 INFO:teuthology.orchestra.run.smithi072.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 191 kB/s | 41 kB 00:00 2024-03-11T19:07:30.902 INFO:teuthology.orchestra.run.smithi072.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 839 kB/s | 238 kB 00:00 2024-03-11T19:07:30.945 INFO:teuthology.orchestra.run.smithi072.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 932 kB/s | 132 kB 00:00 2024-03-11T19:07:30.987 INFO:teuthology.orchestra.run.smithi072.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 211 kB 00:00 2024-03-11T19:07:31.022 INFO:teuthology.orchestra.run.smithi072.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 614 kB/s | 47 kB 00:00 2024-03-11T19:07:31.065 INFO:teuthology.orchestra.run.smithi072.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 557 kB/s | 43 kB 00:00 2024-03-11T19:07:31.302 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-03-11T19:07:31.303 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-03-11T19:07:31.426 INFO:teuthology.orchestra.run.smithi143.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 252 kB/s | 838 kB 00:03 2024-03-11T19:07:31.627 INFO:teuthology.orchestra.run.smithi143.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 217 kB/s | 43 kB 00:00 2024-03-11T19:07:31.794 INFO:teuthology.orchestra.run.smithi143.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-03-11T19:07:31.798 INFO:teuthology.orchestra.run.smithi072.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 53 kB/s | 39 kB 00:00 2024-03-11T19:07:31.902 INFO:teuthology.orchestra.run.smithi143.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-11T19:07:32.383 INFO:teuthology.orchestra.run.smithi072.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-03-11T19:07:32.402 INFO:teuthology.orchestra.run.smithi143.stdout:(118/174): python3-asyncssh-2.13.2-4.el9.noarch 139 kB/s | 548 kB 00:03 2024-03-11T19:07:32.408 INFO:teuthology.orchestra.run.smithi072.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 68 kB/s | 41 kB 00:00 2024-03-11T19:07:32.442 INFO:teuthology.orchestra.run.smithi072.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-03-11T19:07:32.476 INFO:teuthology.orchestra.run.smithi072.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 876 kB/s | 56 kB 00:00 2024-03-11T19:07:32.510 INFO:teuthology.orchestra.run.smithi072.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 948 kB/s | 63 kB 00:00 2024-03-11T19:07:32.535 INFO:teuthology.orchestra.run.smithi072.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 762 kB/s | 44 kB 00:00 2024-03-11T19:07:32.561 INFO:teuthology.orchestra.run.smithi143.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 261 kB/s | 172 kB 00:00 2024-03-11T19:07:32.569 INFO:teuthology.orchestra.run.smithi072.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-11T19:07:32.612 INFO:teuthology.orchestra.run.smithi072.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-11T19:07:32.704 INFO:teuthology.orchestra.run.smithi072.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.1 MB 00:00 2024-03-11T19:07:32.738 INFO:teuthology.orchestra.run.smithi072.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 856 kB/s | 103 kB 00:00 2024-03-11T19:07:32.772 INFO:teuthology.orchestra.run.smithi072.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2024-03-11T19:07:32.807 INFO:teuthology.orchestra.run.smithi072.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-11T19:07:32.883 INFO:teuthology.orchestra.run.smithi072.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-11T19:07:32.950 INFO:teuthology.orchestra.run.smithi072.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-03-11T19:07:33.009 INFO:teuthology.orchestra.run.smithi072.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-03-11T19:07:33.084 INFO:teuthology.orchestra.run.smithi072.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 937 kB/s | 125 kB 00:00 2024-03-11T19:07:33.152 INFO:teuthology.orchestra.run.smithi072.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-03-11T19:07:33.177 INFO:teuthology.orchestra.run.smithi072.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-03-11T19:07:33.278 INFO:teuthology.orchestra.run.smithi072.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-11T19:07:33.337 INFO:teuthology.orchestra.run.smithi072.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-03-11T19:07:33.396 INFO:teuthology.orchestra.run.smithi072.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.7 MB/s | 216 kB 00:00 2024-03-11T19:07:33.530 INFO:teuthology.orchestra.run.smithi072.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-03-11T19:07:33.563 INFO:teuthology.orchestra.run.smithi072.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 209 kB/s | 35 kB 00:00 2024-03-11T19:07:33.597 INFO:teuthology.orchestra.run.smithi072.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 668 kB/s | 44 kB 00:00 2024-03-11T19:07:33.631 INFO:teuthology.orchestra.run.smithi072.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-11T19:07:33.669 INFO:teuthology.orchestra.run.smithi143.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 273 kB/s | 303 kB 00:01 2024-03-11T19:07:33.782 INFO:teuthology.orchestra.run.smithi072.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-03-11T19:07:33.836 INFO:teuthology.orchestra.run.smithi143.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 212 kB/s | 35 kB 00:00 2024-03-11T19:07:33.892 INFO:teuthology.orchestra.run.smithi072.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-03-11T19:07:33.943 INFO:teuthology.orchestra.run.smithi072.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-03-11T19:07:33.993 INFO:teuthology.orchestra.run.smithi072.stdout:(107/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-03-11T19:07:34.019 INFO:teuthology.orchestra.run.smithi072.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-03-11T19:07:34.053 INFO:teuthology.orchestra.run.smithi072.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 794 kB/s | 26 kB 00:00 2024-03-11T19:07:34.087 INFO:teuthology.orchestra.run.smithi072.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-03-11T19:07:34.129 INFO:teuthology.orchestra.run.smithi072.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-11T19:07:34.179 INFO:teuthology.orchestra.run.smithi072.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-11T19:07:34.263 INFO:teuthology.orchestra.run.smithi072.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-03-11T19:07:34.422 INFO:teuthology.orchestra.run.smithi072.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-11T19:07:34.473 INFO:teuthology.orchestra.run.smithi072.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-03-11T19:07:34.507 INFO:teuthology.orchestra.run.smithi072.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-03-11T19:07:34.541 INFO:teuthology.orchestra.run.smithi072.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-03-11T19:07:34.566 INFO:teuthology.orchestra.run.smithi072.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-03-11T19:07:34.602 INFO:teuthology.orchestra.run.smithi072.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-03-11T19:07:34.636 INFO:teuthology.orchestra.run.smithi072.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.1 MB/s | 358 kB 00:00 2024-03-11T19:07:34.678 INFO:teuthology.orchestra.run.smithi072.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-11T19:07:34.704 INFO:teuthology.orchestra.run.smithi072.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-03-11T19:07:34.738 INFO:teuthology.orchestra.run.smithi072.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-11T19:07:34.788 INFO:teuthology.orchestra.run.smithi072.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-03-11T19:07:34.822 INFO:teuthology.orchestra.run.smithi072.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 326 kB/s | 11 kB 00:00 2024-03-11T19:07:34.906 INFO:teuthology.orchestra.run.smithi072.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-03-11T19:07:34.931 INFO:teuthology.orchestra.run.smithi072.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-11T19:07:34.953 INFO:teuthology.orchestra.run.smithi143.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 181 kB/s | 203 kB 00:01 2024-03-11T19:07:34.957 INFO:teuthology.orchestra.run.smithi072.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-03-11T19:07:34.983 INFO:teuthology.orchestra.run.smithi072.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-03-11T19:07:35.008 INFO:teuthology.orchestra.run.smithi072.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-03-11T19:07:35.522 INFO:teuthology.orchestra.run.smithi072.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 1.9 MB/s | 1.0 MB 00:00 2024-03-11T19:07:35.556 INFO:teuthology.orchestra.run.smithi072.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 85 kB/s | 46 kB 00:00 2024-03-11T19:07:35.562 INFO:teuthology.orchestra.run.smithi143.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 113 kB/s | 358 kB 00:03 2024-03-11T19:07:35.590 INFO:teuthology.orchestra.run.smithi072.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-11T19:07:35.650 INFO:teuthology.orchestra.run.smithi072.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 623 kB/s | 58 kB 00:00 2024-03-11T19:07:35.684 INFO:teuthology.orchestra.run.smithi072.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-11T19:07:35.721 INFO:teuthology.orchestra.run.smithi072.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 240 kB/s | 16 kB 00:00 2024-03-11T19:07:35.761 INFO:teuthology.orchestra.run.smithi072.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-11T19:07:35.803 INFO:teuthology.orchestra.run.smithi072.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-03-11T19:07:35.845 INFO:teuthology.orchestra.run.smithi072.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-03-11T19:07:35.913 INFO:teuthology.orchestra.run.smithi072.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 892 kB/s | 59 kB 00:00 2024-03-11T19:07:35.963 INFO:teuthology.orchestra.run.smithi072.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 721 kB/s | 36 kB 00:00 2024-03-11T19:07:35.997 INFO:teuthology.orchestra.run.smithi072.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 977 kB/s | 188 kB 00:00 2024-03-11T19:07:36.039 INFO:teuthology.orchestra.run.smithi072.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-03-11T19:07:36.123 INFO:teuthology.orchestra.run.smithi072.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-11T19:07:36.249 INFO:teuthology.orchestra.run.smithi072.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-03-11T19:07:36.283 INFO:teuthology.orchestra.run.smithi072.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 806 kB/s | 230 kB 00:00 2024-03-11T19:07:36.325 INFO:teuthology.orchestra.run.smithi072.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-03-11T19:07:36.501 INFO:teuthology.orchestra.run.smithi072.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 883 kB/s | 192 kB 00:00 2024-03-11T19:07:36.528 INFO:teuthology.orchestra.run.smithi143.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 149 kB/s | 144 kB 00:00 2024-03-11T19:07:36.612 INFO:teuthology.orchestra.run.smithi143.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-11T19:07:36.635 INFO:teuthology.orchestra.run.smithi072.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-03-11T19:07:36.979 INFO:teuthology.orchestra.run.smithi143.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 48 kB/s | 18 kB 00:00 2024-03-11T19:07:37.288 INFO:teuthology.orchestra.run.smithi143.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 75 kB/s | 23 kB 00:00 2024-03-11T19:07:37.471 INFO:teuthology.orchestra.run.smithi143.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 106 kB/s | 19 kB 00:00 2024-03-11T19:07:37.630 INFO:teuthology.orchestra.run.smithi143.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-03-11T19:07:38.129 INFO:teuthology.orchestra.run.smithi072.stdout:(150/174): librbd1-19.0.0-1607.g94a53970.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-03-11T19:07:38.143 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-03-11T19:07:38.212 INFO:teuthology.orchestra.run.smithi072.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-11T19:07:38.241 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:07:38.284 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:07:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:07:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:07:38.356 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:07:38.371 INFO:teuthology.orchestra.run.smithi072.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 15 MB/s | 2.3 MB 00:00 2024-03-11T19:07:38.409 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:07:38.442 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:07:38.468 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:07:38.472 INFO:teuthology.orchestra.run.smithi072.stdout:(153/174): librados2-19.0.0-1607.g94a53970.el9. 1.7 MB/s | 3.4 MB 00:01 2024-03-11T19:07:38.510 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:07:38.531 INFO:teuthology.orchestra.run.smithi072.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 735 kB/s | 42 kB 00:00 2024-03-11T19:07:38.631 INFO:teuthology.orchestra.run.smithi072.stdout:(155/174): NetworkManager-libnm-1.46.0-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-03-11T19:07:38.665 INFO:teuthology.orchestra.run.smithi072.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-03-11T19:07:38.716 INFO:teuthology.orchestra.run.smithi072.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 857 kB/s | 71 kB 00:00 2024-03-11T19:07:38.741 INFO:teuthology.orchestra.run.smithi072.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 486 kB/s | 36 kB 00:00 2024-03-11T19:07:38.809 INFO:teuthology.orchestra.run.smithi072.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-11T19:07:38.828 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:07:38.851 INFO:teuthology.orchestra.run.smithi072.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-11T19:07:38.886 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:07:38.893 INFO:teuthology.orchestra.run.smithi072.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 276 kB/s | 23 kB 00:00 2024-03-11T19:07:38.935 INFO:teuthology.orchestra.run.smithi072.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-11T19:07:38.988 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:07:38.994 INFO:teuthology.orchestra.run.smithi072.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-11T19:07:39.028 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:07:39.046 INFO:teuthology.orchestra.run.smithi072.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-11T19:07:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 754 kB/s | 64 kB 00:00 2024-03-11T19:07:39.118 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:07:39.122 INFO:teuthology.orchestra.run.smithi072.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 382 kB/s | 29 kB 00:00 2024-03-11T19:07:39.149 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:07:39.185 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:07:39.223 INFO:teuthology.orchestra.run.smithi072.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-11T19:07:39.332 INFO:teuthology.orchestra.run.smithi072.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 10 MB/s | 2.2 MB 00:00 2024-03-11T19:07:39.357 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:07:39.413 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:07:39.441 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:07:39.491 INFO:teuthology.orchestra.run.smithi072.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.6 MB/s | 2.3 MB 00:00 2024-03-11T19:07:39.525 INFO:teuthology.orchestra.run.smithi072.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.4 MB/s | 465 kB 00:00 2024-03-11T19:07:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:07:39.567 INFO:teuthology.orchestra.run.smithi072.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 416 kB/s | 31 kB 00:00 2024-03-11T19:07:39.568 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:07:39.631 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:07:39.667 INFO:teuthology.orchestra.run.smithi072.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-03-11T19:07:39.671 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:07:39.693 INFO:teuthology.orchestra.run.smithi072.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 595 kB/s | 74 kB 00:00 2024-03-11T19:07:39.694 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:07:39.710 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:07:39.861 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:07:39.944 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:07:40.171 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:07:40.329 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:07:40.409 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:07:40.478 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:07:40.566 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:07:40.637 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:07:40.841 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:07:40.951 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:07:41.045 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:07:41.075 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:07:41.102 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:07:41.137 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:07:41.211 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:07:41.265 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:07:41.304 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:07:41.368 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:07:41.402 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:07:41.478 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:07:41.506 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:07:41.584 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:07:41.620 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:07:41.663 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:07:41.692 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:07:41.760 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:07:41.806 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:07:41.879 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:07:42.016 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:07:42.066 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:07:42.265 INFO:teuthology.orchestra.run.smithi143.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 285 kB/s | 2.0 MB 00:07 2024-03-11T19:07:42.496 INFO:teuthology.orchestra.run.smithi143.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 200 kB/s | 46 kB 00:00 2024-03-11T19:07:42.581 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:07:42.661 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:07:42.749 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:07:42.799 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:07:42.808 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:07:42.839 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:07:42.896 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:07:42.934 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:07:42.955 INFO:teuthology.orchestra.run.smithi143.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 172 kB/s | 79 kB 00:00 2024-03-11T19:07:42.963 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:07:43.192 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:07:43.247 INFO:teuthology.orchestra.run.smithi143.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 198 kB/s | 58 kB 00:00 2024-03-11T19:07:43.381 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:07:43.432 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:07:43.476 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:07:43.625 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:07:43.662 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:07:43.717 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:07:43.798 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:07:43.829 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:07:43.866 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:07:43.920 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:07:43.956 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:07:43.999 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:07:44.041 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:07:44.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:07:44.197 INFO:teuthology.orchestra.run.smithi143.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 287 kB/s | 272 kB 00:00 2024-03-11T19:07:44.239 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:07:44.269 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:07:44.297 INFO:teuthology.orchestra.run.smithi143.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-03-11T19:07:44.310 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:07:44.403 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:07:44.443 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:07:44.475 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:07:44.510 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:07:44.534 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:07:44.606 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:07:44.656 INFO:teuthology.orchestra.run.smithi143.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 251 kB/s | 90 kB 00:00 2024-03-11T19:07:44.734 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:07:44.798 INFO:teuthology.orchestra.run.smithi143.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-03-11T19:07:44.833 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:07:44.876 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:07:44.923 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:07:44.942 INFO:teuthology.orchestra.run.smithi143.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-03-11T19:07:44.978 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:07:44.981 INFO:teuthology.orchestra.run.smithi143.stdout:(139/174): python3-kubernetes-26.1.0-2.el9.noar 142 kB/s | 1.0 MB 00:07 2024-03-11T19:07:45.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:07:45.204 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:07:45.332 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:07:45.369 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:07:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:07:45.440 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:07:45.468 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:07:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:07:45.550 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:07:45.582 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:07:45.612 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:07:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 93 kB/s | 59 kB 00:00 2024-03-11T19:07:45.640 INFO:teuthology.orchestra.run.smithi143.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 269 kB/s | 188 kB 00:00 2024-03-11T19:07:45.688 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:07:45.730 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:07:45.764 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:07:45.805 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:07:45.840 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:07:45.866 INFO:teuthology.orchestra.run.smithi143.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 226 kB/s | 51 kB 00:00 2024-03-11T19:07:45.908 INFO:teuthology.orchestra.run.smithi143.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 122 kB/s | 36 kB 00:00 2024-03-11T19:07:45.927 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:07:46.018 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:07:46.061 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:07:46.094 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:07:46.141 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:07:46.238 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:07:46.294 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:07:46.525 INFO:teuthology.orchestra.run.smithi143.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 145 kB/s | 90 kB 00:00 2024-03-11T19:07:46.592 INFO:teuthology.orchestra.run.smithi143.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 317 kB/s | 230 kB 00:00 2024-03-11T19:07:46.611 INFO:teuthology.orchestra.run.smithi072.stdout:(174/174): ceph-test-19.0.0-1607.g94a53970.el9. 1.8 MB/s | 47 MB 00:26 2024-03-11T19:07:46.621 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:07:46.621 INFO:teuthology.orchestra.run.smithi072.stdout:Total 6.3 MB/s | 223 MB 00:35 2024-03-11T19:07:46.675 INFO:teuthology.orchestra.run.smithi143.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-03-11T19:07:46.934 INFO:teuthology.orchestra.run.smithi143.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 742 kB/s | 192 kB 00:00 2024-03-11T19:07:47.022 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:07:47.084 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:07:47.118 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:07:47.165 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:07:47.253 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:07:47.293 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:07:47.470 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:07:47.499 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:07:47.576 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:07:47.709 INFO:teuthology.orchestra.run.smithi143.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 361 kB/s | 427 kB 00:01 2024-03-11T19:07:48.025 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-03-11T19:07:48.213 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:07:48.322 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:07:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-03-11T19:07:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-03-11T19:07:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:(149/174): librados2-19.0.0-1607.g94a53970.el9. 3.0 MB/s | 3.4 MB 00:01 2024-03-11T19:07:49.557 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:07:49.640 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:07:49.680 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:07:49.711 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:07:49.739 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:07:49.793 INFO:teuthology.orchestra.run.smithi143.stdout:(150/174): librbd1-19.0.0-1607.g94a53970.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-03-11T19:07:49.843 INFO:teuthology.orchestra.run.smithi143.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-11T19:07:50.045 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:07:50.069 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:07:50.159 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:07:50.190 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:07:50.226 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:07:50.679 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:07:50.710 INFO:teuthology.orchestra.run.smithi143.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-03-11T19:07:50.721 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:07:50.776 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:07:50.806 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:07:50.818 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-03-11T19:07:50.819 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-03-11T19:07:51.244 INFO:teuthology.orchestra.run.smithi143.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-03-11T19:07:51.344 INFO:teuthology.orchestra.run.smithi143.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 425 kB/s | 42 kB 00:00 2024-03-11T19:07:51.611 INFO:teuthology.orchestra.run.smithi143.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 931 kB/s | 248 kB 00:00 2024-03-11T19:07:51.736 INFO:teuthology.orchestra.run.smithi143.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 571 kB/s | 71 kB 00:00 2024-03-11T19:07:51.836 INFO:teuthology.orchestra.run.smithi143.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 365 kB/s | 36 kB 00:00 2024-03-11T19:07:52.103 INFO:teuthology.orchestra.run.smithi143.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 721 kB/s | 192 kB 00:00 2024-03-11T19:07:52.462 INFO:teuthology.orchestra.run.smithi143.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 714 kB/s | 256 kB 00:00 2024-03-11T19:07:52.512 INFO:teuthology.orchestra.run.smithi143.stdout:(160/174): thrift-0.15.0-2.el9.x86_64.rpm 290 kB/s | 1.6 MB 00:05 2024-03-11T19:07:52.562 INFO:teuthology.orchestra.run.smithi143.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-11T19:07:52.746 INFO:teuthology.orchestra.run.smithi143.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 845 kB/s | 155 kB 00:00 2024-03-11T19:07:52.929 INFO:teuthology.orchestra.run.smithi143.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 740 kB/s | 136 kB 00:00 2024-03-11T19:07:52.955 INFO:teuthology.orchestra.run.smithi143.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 245 kB/s | 108 kB 00:00 2024-03-11T19:07:53.002 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:07:53.038 INFO:teuthology.orchestra.run.smithi143.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 588 kB/s | 64 kB 00:00 2024-03-11T19:07:53.053 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:07:53.089 INFO:teuthology.orchestra.run.smithi143.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 215 kB/s | 29 kB 00:00 2024-03-11T19:07:53.555 INFO:teuthology.orchestra.run.smithi143.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.6 MB/s | 2.2 MB 00:00 2024-03-11T19:07:53.590 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:07:53.631 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:07:53.703 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:07:53.704 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:07:53.704 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:07:53.704 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:07:53.732 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:07:54.556 INFO:teuthology.orchestra.run.smithi143.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 817 kB/s | 1.2 MB 00:01 2024-03-11T19:07:54.723 INFO:teuthology.orchestra.run.smithi143.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 2.3 MB 00:01 2024-03-11T19:07:54.833 INFO:teuthology.orchestra.run.smithi143.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 281 kB/s | 31 kB 00:00 2024-03-11T19:07:55.034 INFO:teuthology.orchestra.run.smithi143.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 973 kB/s | 465 kB 00:00 2024-03-11T19:07:55.142 INFO:teuthology.orchestra.run.smithi143.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 687 kB/s | 74 kB 00:00 2024-03-11T19:07:55.784 INFO:teuthology.orchestra.run.smithi143.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 1.1 MB 00:00 2024-03-11T19:07:58.302 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-03-11T19:07:58.444 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:07:58.484 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:07:58.523 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:07:58.566 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:07:58.663 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:07:58.767 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:07:58.824 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:07:58.850 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:07:58.926 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:07:59.343 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:07:59.468 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:07:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:07:59.661 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:07:59.787 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:07:59.826 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:07:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:08:00.239 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:08:00.296 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:08:00.324 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:00.467 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:00.493 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:08:00.584 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:08:00.638 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:08:00.671 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:08:00.742 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:08:01.045 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:08:01.099 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:08:01.138 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:08:01.200 INFO:teuthology.orchestra.run.smithi143.stdout:(174/174): libarrow-9.0.0-10.el9.x86_64.rpm 128 kB/s | 4.4 MB 00:35 2024-03-11T19:08:01.204 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:08:01.204 INFO:teuthology.orchestra.run.smithi143.stdout:Total 4.3 MB/s | 223 MB 00:51 2024-03-11T19:08:01.655 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-03-11T19:08:01.743 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:08:01.812 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:08:01.903 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:08:01.927 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-03-11T19:08:01.927 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-03-11T19:08:02.195 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:02.243 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:02.608 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:02.795 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:08:02.920 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:08:02.969 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:08:03.003 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:08:03.069 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:08:03.187 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:08:03.253 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:08:03.307 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:08:03.409 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:08:03.455 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:08:03.519 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:08:03.561 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:08:03.602 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:08:03.638 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-03-11T19:08:03.638 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-03-11T19:08:03.720 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:08:03.759 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:08:03.836 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:08:03.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:08:03.985 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:08:04.046 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:08:04.156 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:08:04.336 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:08:04.401 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:08:05.095 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:08:05.196 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-03-11T19:08:05.228 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:05.318 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:08:05.365 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:08:05.388 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:08:05.393 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:05.409 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:05.409 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:05.409 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:08:05.409 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:05.429 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:08:05.498 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:08:05.548 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:08:05.593 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:08:05.985 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:08:06.198 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:08:06.292 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:08:06.387 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:08:06.599 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:08:06.653 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:08:06.729 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:08:06.822 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:08:06.871 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:08:06.872 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:06.897 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:06.898 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:06.898 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:08:06.898 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:06.898 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:06.898 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:06.920 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:08:06.990 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:08:07.039 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:08:07.106 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:08:07.106 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:07.129 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:07.129 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:07.130 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:08:07.130 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:07.130 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:07.130 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:07.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:08:07.382 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:08:07.440 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:08:07.489 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:08:07.548 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:08:07.567 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:07.590 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:07.590 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:07.590 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:08:07.591 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:07.591 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:07.591 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:07.687 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:08:07.702 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:08:07.730 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:08:07.777 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:08:07.814 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:08:07.849 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:08:08.006 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:08:08.078 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:08.109 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:08.140 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:08:08.183 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:08:08.201 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:08:08.232 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:08.322 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:08:08.375 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:08:08.437 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:08:08.463 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:08.500 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:08.511 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:08:08.774 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:08:08.843 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:08:09.016 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:08:09.067 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:08:09.108 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:08:09.147 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:08:09.197 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:08:09.245 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:08:09.304 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:08:09.362 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:08:09.415 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:08:09.458 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:09.491 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:09.551 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:08:09.592 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:09.593 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:08:09.628 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:08:09.690 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:08:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:08:09.762 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:08:09.794 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:09.817 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:09.817 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:09.817 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:08:09.818 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:09.818 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:09.818 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:09.861 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:09.872 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:09.878 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:08:09.916 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:08:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:08:10.050 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:08:10.087 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:08:10.144 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:08:10.319 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:08:10.381 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:08:10.487 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-03-11T19:08:10.587 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:08:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:08:10.652 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:08:10.661 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:08:10.710 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:08:10.772 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:08:10.772 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:08:10.796 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:10.806 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:08:10.819 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:10.819 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:10.819 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:08:10.819 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:10.819 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:10.820 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:08:10.863 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:10.886 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:10.886 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:10.886 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:08:10.886 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:10.890 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:08:10.931 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:10.954 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:11.208 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:08:11.273 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:11.274 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:08:11.296 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:11.296 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:11.296 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:08:11.296 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:11.296 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:11.297 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:11.339 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:08:11.396 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:08:11.410 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:08:11.434 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:08:11.488 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:08:11.525 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:08:11.551 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:08:11.554 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:08:11.599 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:08:11.671 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:08:11.714 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:08:11.788 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:08:11.836 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:08:11.865 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:08:11.968 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:11.980 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:08:11.991 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:08:12.055 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:08:12.079 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:08:12.103 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:08:12.126 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:08:12.141 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:08:12.301 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:08:12.332 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:08:12.372 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:08:12.476 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:08:12.566 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:08:12.643 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:08:12.713 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:12.770 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:12.858 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:08:12.998 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:08:13.017 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:08:13.130 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:08:13.220 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:08:13.258 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:08:13.285 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:08:13.330 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:08:13.434 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:08:13.684 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:08:13.812 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:08:13.876 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:08:13.911 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:08:13.955 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:08:13.987 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:08:14.015 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:08:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:08:14.129 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:08:14.172 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:08:14.201 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:08:14.269 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:08:14.316 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:08:14.388 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:08:14.492 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:08:14.523 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:08:14.576 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:08:14.599 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:08:14.638 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:08:14.679 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:08:14.720 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:08:15.095 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:08:15.100 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:08:15.128 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:08:15.179 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:08:15.239 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:08:15.252 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:08:15.269 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:08:15.309 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:08:15.317 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:08:15.349 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:08:15.359 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:08:15.390 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:08:15.445 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:08:15.474 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:08:15.689 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:08:15.719 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:08:15.761 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:08:15.791 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:08:15.818 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:08:15.878 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:08:15.880 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:08:15.908 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:08:15.940 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:08:15.984 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:08:16.000 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:08:16.001 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:08:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:08:16.117 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:08:16.124 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:08:16.158 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:08:16.171 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:08:16.197 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:08:16.225 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:08:16.258 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:08:16.292 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:08:16.298 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:08:16.325 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:08:16.338 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:08:16.357 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:08:16.374 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:08:16.387 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:08:16.421 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:08:16.441 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:08:16.458 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:08:16.502 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:08:16.535 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:08:16.540 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:08:16.584 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:08:16.716 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:08:16.765 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:08:16.795 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:08:16.837 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:08:16.930 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:08:16.971 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:08:17.010 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:08:17.037 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:08:17.061 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:08:17.089 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:17.124 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:17.133 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:08:17.144 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:17.261 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:08:17.360 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:08:17.403 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:08:17.452 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:08:17.515 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:08:17.707 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:08:17.750 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:08:17.877 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:08:17.922 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:08:17.960 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:08:17.971 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:08:18.002 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:08:18.031 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:08:18.053 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:08:18.065 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:08:18.101 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:08:18.104 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:08:18.136 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:08:18.166 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:08:18.182 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:08:18.231 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:08:18.242 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:08:18.256 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:08:18.276 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:08:18.281 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:08:18.310 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:08:18.316 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:08:18.316 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:18.359 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:08:18.394 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:08:18.404 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:18.434 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:08:18.434 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:18.482 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:08:18.518 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:18.544 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:08:18.570 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:18.572 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:08:18.577 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:08:18.603 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:08:18.617 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:08:18.627 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:18.628 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:08:18.648 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:08:18.652 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:08:18.692 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:08:18.704 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:08:18.724 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:08:18.750 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:08:18.775 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:08:18.800 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:08:18.808 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:08:18.825 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:08:18.832 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:18.832 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:18.867 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:08:19.012 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:19.012 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:19.095 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:19.119 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:08:19.144 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:08:19.177 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:08:19.204 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:08:19.262 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:19.268 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:19.310 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:08:19.487 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:08:19.487 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:08:19.487 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:08:19.488 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:19.528 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:08:19.598 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:08:19.648 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:08:19.681 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:08:19.738 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:08:19.834 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:08:19.866 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:08:20.044 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:08:20.072 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:08:20.150 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:08:20.784 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:08:20.894 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:08:22.094 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:08:22.182 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:08:22.213 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:08:22.245 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:08:22.271 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:08:22.579 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:08:22.610 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:08:22.692 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:08:22.724 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:08:22.768 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:08:23.224 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:08:23.255 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:08:23.310 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:08:23.340 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:08:24.048 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:08:24.049 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:08:24.050 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:08:24.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:08:24.052 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:08:24.053 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:08:24.054 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:08:24.055 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:08:24.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:08:24.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:08:24.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:08:24.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:08:24.060 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:08:24.061 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:08:24.062 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:08:25.512 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:25.545 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:26.082 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:08:26.132 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:08:26.213 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:08:26.213 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:08:26.213 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:08:26.213 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:26.308 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:08:27.034 INFO:teuthology.orchestra.run.smithi106.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:08:27.035 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.036 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.037 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:08:27.038 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.039 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:08:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:08:27.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:08:27.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:08:27.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:08:27.046 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:08:27.047 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:08:27.047 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-11T19:08:27.047 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-03-11T19:08:27.526 DEBUG:teuthology.parallel:result is None 2024-03-11T19:08:35.402 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-03-11T19:08:35.403 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:35.640 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:35.693 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:35.693 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:35.693 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:08:35.693 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:37.348 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:37.454 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:37.454 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:37.454 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:08:37.454 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:37.455 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:37.455 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:37.690 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:37.763 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-03-11T19:08:38.241 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:38.242 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:38.313 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:38.407 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:38.422 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:08:38.423 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:38.423 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:08:38.423 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:38.430 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:08:39.103 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:39.134 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:39.184 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:08:39.287 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:08:39.316 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:39.642 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:39.684 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:39.912 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:08:39.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:40.163 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:40.185 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:08:40.185 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:40.186 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:08:40.186 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:40.186 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:08:40.186 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:40.622 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:08:40.644 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:40.740 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:08:40.808 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:40.975 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:41.143 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:08:41.183 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:41.191 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:41.214 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:08:41.254 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:08:41.255 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:41.256 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:41.256 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:08:41.256 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:41.256 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:41.256 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:41.314 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:08:41.317 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:41.355 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:41.445 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:08:41.567 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:08:41.605 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:42.449 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:08:42.479 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:42.558 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:42.561 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:08:42.589 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:42.614 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:42.689 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:08:42.750 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:42.825 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:42.858 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:08:42.890 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:42.913 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:08:42.913 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:42.913 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:08:42.914 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:42.914 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:08:42.914 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:42.958 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:42.968 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:08:43.015 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:08:43.193 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:43.262 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:43.262 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:43.262 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:08:43.262 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:43.262 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:43.263 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:43.869 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:08:43.911 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:43.934 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:08:43.934 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:43.934 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:08:43.934 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:43.934 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:08:43.935 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:43.978 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:44.000 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:08:44.000 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:44.000 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:08:44.000 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:44.054 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:08:44.076 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:44.388 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:44.410 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:08:44.410 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:08:44.411 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:08:44.411 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:44.411 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:08:44.411 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:08:48.040 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:08:48.095 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:08:48.131 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:08:48.169 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:08:48.438 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:08:48.574 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:08:48.623 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:08:48.685 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:08:48.742 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:08:48.788 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:08:48.817 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:08:48.858 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:08:48.873 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:08:48.889 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:08:48.916 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:08:48.921 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:08:48.973 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:08:49.059 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:08:49.095 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:08:49.192 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:08:49.215 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:08:49.225 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:08:49.265 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:08:49.312 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:08:49.381 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:08:49.416 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:08:49.441 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:08:49.473 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:08:49.503 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:08:49.567 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:08:49.617 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:08:49.643 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:08:50.080 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:50.173 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:50.241 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:50.242 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:50.261 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:08:51.070 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:08:51.146 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:08:51.186 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:08:51.267 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:08:51.300 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:08:51.325 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:08:51.350 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:08:51.385 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:08:51.385 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:51.503 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:08:51.504 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:51.583 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:08:51.588 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:51.614 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:08:51.647 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:08:51.673 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:08:51.692 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:08:51.722 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:08:51.752 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:08:51.755 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:08:51.795 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:08:51.827 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:08:51.853 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:08:51.878 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:08:51.891 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:08:51.904 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:08:51.929 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:08:51.936 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:51.936 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:51.939 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:08:51.974 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:08:52.010 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:08:52.078 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:08:52.078 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:52.132 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:52.132 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:52.215 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:52.231 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:08:52.240 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:08:52.265 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:08:52.282 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:08:52.282 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:52.307 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:08:52.333 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:08:52.397 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:52.431 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:08:52.474 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:08:52.522 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:08:52.564 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:08:52.597 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:08:52.694 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:08:52.783 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:08:52.841 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:08:52.883 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:08:52.917 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:08:52.950 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:08:52.984 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:08:53.021 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:53.021 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:53.352 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:08:53.352 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:53.485 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:08:53.516 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:08:53.558 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:08:53.614 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:08:53.651 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:08:53.770 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:54.033 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:54.033 INFO:teuthology.orchestra.run.smithi072.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-11T19:08:54.034 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:08:57.239 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:08:57.240 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:08:57.241 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:08:57.241 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:08:57.241 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:08:57.241 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:08:57.241 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:08:57.256 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:08:57.257 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:08:57.258 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:08:57.259 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:08:57.260 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:08:57.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:08:57.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:08:57.263 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:08:57.264 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:08:57.265 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:08:57.266 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:08:57.267 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:09:00.244 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:09:00.244 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.245 INFO:teuthology.orchestra.run.smithi143.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-03-11T19:09:00.246 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.247 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:09:00.248 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.249 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:09:00.250 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:09:00.251 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:09:00.252 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:09:00.253 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:09:00.254 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:09:00.255 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.256 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:09:00.257 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-03-11T19:09:00.729 DEBUG:teuthology.parallel:result is None 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:09:02.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:09:02.688 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:09:02.689 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:09:02.705 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:09:02.705 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:09:02.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:09:02.707 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:09:02.708 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:09:02.709 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:09:02.710 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:09:02.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:09:02.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:09:02.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:09:02.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:09:02.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:09:02.716 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:09:02.717 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:09:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:09:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:09:05.254 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-03-11T19:09:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:09:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:09:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:09:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:09:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:09:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:09:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:09:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:09:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:09:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:09:05.268 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-03-11T19:09:06.100 DEBUG:teuthology.parallel:result is None 2024-03-11T19:09:06.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:06.409 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:09:06.469 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:09:06.470 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:09:06.470 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:09:06.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:06.751 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:09:06.778 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:09:06.778 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:09:06.779 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:09:06.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:07.086 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:09:07.112 INFO:teuthology.orchestra.run.smithi143.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:09:07.112 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:09:07.112 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:09:07.115 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-11T19:09:07.115 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:09:07.115 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:09:07.194 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:09:07.194 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:09:07.229 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:09:07.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:09:07.263 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-11T19:09:07.264 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:09:07.264 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:09:07.342 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:09:07.456 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:09:07.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:09:07.488 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:09:07.564 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:09:07.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:09:07.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:09:07.665 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-11T19:09:07.665 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:09:07.665 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:09:07.755 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:09:07.894 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:09:07.894 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:09:07.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:09:07.995 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:09:07.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:09:08.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:09:08.097 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-11T19:09:08.097 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:09:08.097 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:09:08.198 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:09:08.284 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:09:08.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:09:08.316 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:09:08.386 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:09:08.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:09:08.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:09:08.488 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-11T19:09:08.590 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-11T19:09:08.591 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:08.591 INFO:tasks.cephadm:Cluster fsid is d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:09:08.591 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-11T19:09:08.592 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.72', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.143'} 2024-03-11T19:09:08.592 INFO:tasks.cephadm:First mon is mon.a on smithi072 2024-03-11T19:09:08.592 INFO:tasks.cephadm:First mgr is x 2024-03-11T19:09:08.592 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-11T19:09:08.592 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-03-11T19:09:08.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-03-11T19:09:08.723 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-03-11T19:09:08.754 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-11T19:09:08.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:08.898 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77431/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:08:19.449750', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77429/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:06:59.696700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-11T19:09:09.179 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 94a539707733325322964c6a187c6a1b41be1daf from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:09:09.180 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm 2024-03-11T19:09:09.181 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:09.690 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:09 /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:09.690 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.390 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:09 /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.390 DEBUG:teuthology.orchestra.run.smithi143:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.804 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:09 /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.804 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.864 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.881 DEBUG:teuthology.orchestra.run.smithi143:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:09:10.915 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf on all hosts... 2024-03-11T19:09:10.915 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:09:10.923 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:09:10.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:09:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:09:11.842 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:09:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:09:53.975 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:09:53.976 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:10:05.622 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-03-11T19:10:05.623 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-11T19:10:09.559 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-03-11T19:10:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:10:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:10:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-03-11T19:10:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:10:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-03-11T19:10:09.560 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-03-11T19:10:09.584 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-03-11T19:10:09.685 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-03-11T19:10:09.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-03-11T19:10:09.751 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-03-11T19:10:09.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-03-11T19:10:09.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-03-11T19:10:09.933 INFO:tasks.cephadm:Writing seed config... 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-11T19:10:09.935 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-11T19:10:09.936 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-11T19:10:09.937 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-11T19:10:09.938 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-11T19:10:09.939 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:10:09.939 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-11T19:10:09.993 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d6472314-dfda-11ee-95c8-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-11T19:10:09.993 DEBUG:teuthology.orchestra.run.smithi072:mon.a> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service 2024-03-11T19:10:10.041 DEBUG:teuthology.orchestra.run.smithi072:mgr.x> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x.service 2024-03-11T19:10:10.086 INFO:tasks.cephadm:Bootstrapping... 2024-03-11T19:10:10.086 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf -v bootstrap --fsid d6472314-dfda-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.72 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:10:10.353 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:10:10.353 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf', '-v', 'bootstrap', '--fsid', 'd6472314-dfda-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.72', '--skip-admin-label'] 2024-03-11T19:10:10.407 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:10:10.408 INFO:teuthology.orchestra.run.smithi072.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-11T19:10:10.408 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying podman|docker is present... 2024-03-11T19:10:10.455 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:10:10.456 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying lvm2 is present... 2024-03-11T19:10:10.456 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying time synchronization is in place... 2024-03-11T19:10:10.471 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:10:10.471 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:10:10.488 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:10:10.488 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:10:10.511 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-03-11T19:10:10.534 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-03-11T19:10:11.116 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:10:11.116 INFO:teuthology.orchestra.run.smithi072.stdout:Repeating the final host check... 2024-03-11T19:10:11.185 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:10:11.185 INFO:teuthology.orchestra.run.smithi072.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-11T19:10:11.185 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl is present 2024-03-11T19:10:11.185 INFO:teuthology.orchestra.run.smithi072.stdout:lvcreate is present 2024-03-11T19:10:11.198 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:10:11.199 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:10:11.217 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:10:11.217 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:10:11.237 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-03-11T19:10:11.255 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-03-11T19:10:11.256 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:10:11.256 INFO:teuthology.orchestra.run.smithi072.stdout:Host looks OK 2024-03-11T19:10:11.256 INFO:teuthology.orchestra.run.smithi072.stdout:Cluster fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:11.257 INFO:teuthology.orchestra.run.smithi072.stdout:Acquiring lock 140405547610064 on /run/cephadm/d6472314-dfda-11ee-95c8-87774f69a715.lock 2024-03-11T19:10:11.257 INFO:teuthology.orchestra.run.smithi072.stdout:Lock 140405547610064 acquired on /run/cephadm/d6472314-dfda-11ee-95c8-87774f69a715.lock 2024-03-11T19:10:11.259 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 3300 ... 2024-03-11T19:10:11.260 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 6789 ... 2024-03-11T19:10:11.260 INFO:teuthology.orchestra.run.smithi072.stdout:Base mon IP(s) is [172.21.15.72:3300, 172.21.15.72:6789], mon addrv is [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-03-11T19:10:11.275 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.72 metric 100 2024-03-11T19:10:11.276 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.72 metric 100 2024-03-11T19:10:11.289 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-11T19:10:11.289 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-11T19:10:11.289 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1728sec hoplimit 64 pref medium 2024-03-11T19:10:11.300 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-11T19:10:11.300 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-11T19:10:11.300 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:10:11.301 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-11T19:10:11.301 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2024-03-11T19:10:11.301 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:10:11.303 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-03-11T19:10:11.303 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-03-11T19:10:11.303 INFO:teuthology.orchestra.run.smithi072.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-11T19:10:11.306 INFO:teuthology.orchestra.run.smithi072.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-11T19:10:11.309 INFO:teuthology.orchestra.run.smithi072.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:10:12.497 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:10:12.497 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:10:12.497 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Getting image source signatures 2024-03-11T19:10:12.497 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:0b3c7bee4e66dab4b5376aaf0baf8ba5fc1c3e66c27dc601157782d617bd9dc6 2024-03-11T19:10:12.497 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-11T19:10:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying config sha256:a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:10:12.498 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-11T19:10:13.648 INFO:teuthology.orchestra.run.smithi072.stdout:ceph: stdout ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:10:13.648 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph version: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:10:13.648 INFO:teuthology.orchestra.run.smithi072.stdout:Extracting ceph user uid/gid from container image... 2024-03-11T19:10:14.758 INFO:teuthology.orchestra.run.smithi072.stdout:stat: stdout 167 167 2024-03-11T19:10:14.758 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial keys... 2024-03-11T19:10:15.866 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQAXV+9lETySDBAAqQmtpYDetccwC3TWPLgQuA== 2024-03-11T19:10:16.958 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQAYV+9l1aozFBAAaHcPJuE5k6RxIj/fFmt2Kg== 2024-03-11T19:10:18.799 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQAZV+9lkSiAGBAAI19+7xU/iaRBdOzKHZT+lw== 2024-03-11T19:10:18.800 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial monmap... 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool for a [v2:172.21.15.72:3300,v1:172.21.15.72:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = quincy 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: set fsid to d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:10:19.889 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:10:19.890 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mon... 2024-03-11T19:10:21.042 INFO:teuthology.orchestra.run.smithi072.stdout:create mon.a on 2024-03-11T19:10:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-11T19:10:21.631 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-11T19:10:21.923 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target. 2024-03-11T19:10:21.923 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target. 2024-03-11T19:10:22.306 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a 2024-03-11T19:10:22.306 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service: Unit ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service not loaded. 2024-03-11T19:10:22.588 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target.wants/ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715@.service. 2024-03-11T19:10:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:23 smithi072 podman[31302]: 2024-03-11 19:10:23.016834448 +0000 UTC m=+0.152166750 container create 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-11T19:10:24.391 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:10:24.391 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-03-11T19:10:24.408 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:10:24.408 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:10:24.408 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-03-11T19:10:24.408 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:10:24.409 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon to start... 2024-03-11T19:10:24.409 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon... 2024-03-11T19:10:24.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 podman[31302]: 2024-03-11 19:10:24.263645867 +0000 UTC m=+1.398978171 container init 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:10:24.562 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 podman[31302]: 2024-03-11 19:10:24.274882103 +0000 UTC m=+1.410214431 container start 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T19:10:24.562 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 ceph-mon[31321]: mkfs d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:24.562 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 bash[31302]: 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca 2024-03-11T19:10:24.562 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 ceph-mon[31321]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:10:24.562 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:24 smithi072 systemd[1]: Started Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: monmap e1: 1 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: fsmap 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: osdmap e1: 0 total, 0 up, 0 in 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: mgrmap e1: no daemons active 2024-03-11T19:10:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:25 smithi072 ceph-mon[31321]: from='client.? 172.21.15.72:0/3260374707' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-11T19:10:25.742 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout cluster: 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout id: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout services: 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.73282s) 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:25.749 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout data: 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pgs: 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.988+0000 7f4d68600700 1 Processor -- start 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.988+0000 7f4d68600700 1 -- start start 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.988+0000 7f4d68600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.988+0000 7f4d68600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d601080b0 con 0x7f4d601076c0 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.989+0000 7f4d65c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.990+0000 7f4d65c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33554/0 (socket says 172.21.15.72:33554) 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.990+0000 7f4d65c00700 1 -- 172.21.15.72:0/2864827477 learned_addr learned my addr 172.21.15.72:0/2864827477 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.990+0000 7f4d65c00700 1 -- 172.21.15.72:0/2864827477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60108870 con 0x7f4d601076c0 2024-03-11T19:10:25.750 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.990+0000 7f4d65c00700 1 --2- 172.21.15.72:0/2864827477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176ec5707517bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.990+0000 7f4d5fe00700 1 -- 172.21.15.72:0/2864827477 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54004160 con 0x7f4d601076c0 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.991+0000 7f4d5fe00700 1 -- 172.21.15.72:0/2864827477 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d540042c0 con 0x7f4d601076c0 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.991+0000 7f4d5fe00700 1 -- 172.21.15.72:0/2864827477 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540045a0 con 0x7f4d601076c0 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.991+0000 7f4d68600700 1 -- 172.21.15.72:0/2864827477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 msgr2=0x7f4d60107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.991+0000 7f4d68600700 1 --2- 172.21.15.72:0/2864827477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.991+0000 7f4d68600700 1 -- 172.21.15.72:0/2864827477 shutdown_connections 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.992+0000 7f4d68600700 1 --2- 172.21.15.72:0/2864827477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d60107ae0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.992+0000 7f4d68600700 1 -- 172.21.15.72:0/2864827477 >> 172.21.15.72:0/2864827477 conn(0x7f4d60102d00 msgr2=0x7f4d60105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.992+0000 7f4d68600700 1 -- 172.21.15.72:0/2864827477 shutdown_connections 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.992+0000 7f4d68600700 1 -- 172.21.15.72:0/2864827477 wait complete. 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.993+0000 7f4d68600700 1 Processor -- start 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.993+0000 7f4d68600700 1 -- start start 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.993+0000 7f4d68600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:25.751 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.993+0000 7f4d68600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6019d360 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.994+0000 7f4d65c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.994+0000 7f4d65c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33560/0 (socket says 172.21.15.72:33560) 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.994+0000 7f4d65c00700 1 -- 172.21.15.72:0/3260374707 learned_addr learned my addr 172.21.15.72:0/3260374707 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.994+0000 7f4d65c00700 1 -- 172.21.15.72:0/3260374707 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009550 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.994+0000 7f4d65c00700 1 --2- 172.21.15.72:0/3260374707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4d54004000 tx=0x7f4d54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54011630 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54011790 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54011a30 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6019d560 con 0x7f4d601076c0 2024-03-11T19:10:25.752 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6019da00 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d54021070 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.995+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4d5401b820 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.996+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d60196be0 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:24.999+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d5401ba80 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.079+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4d60108d90 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.080+0000 7f4d5e000700 1 -- 172.21.15.72:0/3260374707 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f4d5401bc20 con 0x7f4d601076c0 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 msgr2=0x7f4d6019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 --2- 172.21.15.72:0/3260374707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4d54004000 tx=0x7f4d54004750 comp rx=0 tx=0).stop 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 shutdown_connections 2024-03-11T19:10:25.753 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 --2- 172.21.15.72:0/3260374707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d601076c0 0x7f4d6019ce20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:25.754 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 >> 172.21.15.72:0/3260374707 conn(0x7f4d60102d00 msgr2=0x7f4d60193d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:25.755 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 shutdown_connections 2024-03-11T19:10:25.755 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:25.082+0000 7f4d68600700 1 -- 172.21.15.72:0/3260374707 wait complete. 2024-03-11T19:10:25.755 INFO:teuthology.orchestra.run.smithi072.stdout:mon is available 2024-03-11T19:10:25.755 INFO:teuthology.orchestra.run.smithi072.stdout:Assimilating anything we can from ceph.conf... 2024-03-11T19:10:26.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:26 smithi072 ceph-mon[31321]: from='client.? 172.21.15.72:0/3436480690' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-11T19:10:26.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:26 smithi072 ceph-mon[31321]: from='client.? 172.21.15.72:0/3436480690' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-11T19:10:27.084 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:27.084 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:10:27.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:10:27.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.338+0000 7f11e8800700 1 Processor -- start 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.338+0000 7f11e8800700 1 -- start start 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.339+0000 7f11e8800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.339+0000 7f11e8800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e007b8e0 con 0x7f11e007ce60 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.339+0000 7f11e5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.339+0000 7f11e5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33570/0 (socket says 172.21.15.72:33570) 2024-03-11T19:10:27.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.339+0000 7f11e5e00700 1 -- 172.21.15.72:0/1661848215 learned_addr learned my addr 172.21.15.72:0/1661848215 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.340+0000 7f11e5e00700 1 -- 172.21.15.72:0/1661848215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e007ba20 con 0x7f11e007ce60 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.340+0000 7f11e5e00700 1 --2- 172.21.15.72:0/1661848215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098a0 tx=0x7f11d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0733843f75856e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.340+0000 7f11e4a00700 1 -- 172.21.15.72:0/1661848215 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d4004160 con 0x7f11e007ce60 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e4a00700 1 -- 172.21.15.72:0/1661848215 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d40042c0 con 0x7f11e007ce60 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e4a00700 1 -- 172.21.15.72:0/1661848215 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d40045a0 con 0x7f11e007ce60 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 -- 172.21.15.72:0/1661848215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 msgr2=0x7f11e007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 --2- 172.21.15.72:0/1661848215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098a0 tx=0x7f11d4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 -- 172.21.15.72:0/1661848215 shutdown_connections 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 --2- 172.21.15.72:0/1661848215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e007b310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 -- 172.21.15.72:0/1661848215 >> 172.21.15.72:0/1661848215 conn(0x7f11e0102cd0 msgr2=0x7f11e01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 -- 172.21.15.72:0/1661848215 shutdown_connections 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.341+0000 7f11e8800700 1 -- 172.21.15.72:0/1661848215 wait complete. 2024-03-11T19:10:27.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e8800700 1 Processor -- start 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e8800700 1 -- start start 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e8800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e8800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e019d300 con 0x7f11e007ce60 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33586/0 (socket says 172.21.15.72:33586) 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.343+0000 7f11e5e00700 1 -- 172.21.15.72:0/3436480690 learned_addr learned my addr 172.21.15.72:0/3436480690 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11e5e00700 1 -- 172.21.15.72:0/3436480690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4009550 con 0x7f11e007ce60 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11e5e00700 1 --2- 172.21.15.72:0/3436480690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f11e007c380 tx=0x7f11d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:27.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d4011630 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4011790 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d4011a30 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.344+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e019d500 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.345+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e019d9a0 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.345+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f11d4021070 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.345+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f11d401b7d0 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.346+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e0196c70 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.348+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f11d401b990 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.424+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f11e007bd90 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.433+0000 7f11daa00700 1 -- 172.21.15.72:0/3436480690 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f11d401bb30 con 0x7f11e007ce60 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 msgr2=0x7f11e019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 --2- 172.21.15.72:0/3436480690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f11e007c380 tx=0x7f11d4004750 comp rx=0 tx=0).stop 2024-03-11T19:10:27.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 shutdown_connections 2024-03-11T19:10:27.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 --2- 172.21.15.72:0/3436480690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11e007ce60 0x7f11e019cdc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:27.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 >> 172.21.15.72:0/3436480690 conn(0x7f11e0102cd0 msgr2=0x7f11e0103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:27.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.434+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 shutdown_connections 2024-03-11T19:10:27.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:26.435+0000 7f11e8800700 1 -- 172.21.15.72:0/3436480690 wait complete. 2024-03-11T19:10:27.096 INFO:teuthology.orchestra.run.smithi072.stdout:Generating new minimal ceph.conf... 2024-03-11T19:10:27.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:27 smithi072 ceph-mon[31321]: from='client.? 172.21.15.72:0/3243995817' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.653+0000 7fe7a1200700 1 Processor -- start 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.653+0000 7fe7a1200700 1 -- start start 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.654+0000 7fe7a1200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.654+0000 7fe7a1200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1080b0 con 0x7fe79c1076c0 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.654+0000 7fe799e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.654+0000 7fe799e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33592/0 (socket says 172.21.15.72:33592) 2024-03-11T19:10:28.460 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.654+0000 7fe799e00700 1 -- 172.21.15.72:0/846251416 learned_addr learned my addr 172.21.15.72:0/846251416 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.655+0000 7fe799e00700 1 -- 172.21.15.72:0/846251416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c108870 con 0x7fe79c1076c0 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe799e00700 1 --2- 172.21.15.72:0/846251416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abc8484b24fc11e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe798a00700 1 -- 172.21.15.72:0/846251416 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe78c004160 con 0x7fe79c1076c0 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe798a00700 1 -- 172.21.15.72:0/846251416 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe78c0042c0 con 0x7fe79c1076c0 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe798a00700 1 -- 172.21.15.72:0/846251416 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe78c011ac0 con 0x7fe79c1076c0 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe7a1200700 1 -- 172.21.15.72:0/846251416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 msgr2=0x7fe79c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.656+0000 7fe7a1200700 1 --2- 172.21.15.72:0/846251416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.657+0000 7fe7a1200700 1 -- 172.21.15.72:0/846251416 shutdown_connections 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.657+0000 7fe7a1200700 1 --2- 172.21.15.72:0/846251416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c107ae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.657+0000 7fe7a1200700 1 -- 172.21.15.72:0/846251416 >> 172.21.15.72:0/846251416 conn(0x7fe79c102d00 msgr2=0x7fe79c105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.657+0000 7fe7a1200700 1 -- 172.21.15.72:0/846251416 shutdown_connections 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.657+0000 7fe7a1200700 1 -- 172.21.15.72:0/846251416 wait complete. 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.658+0000 7fe7a1200700 1 Processor -- start 2024-03-11T19:10:28.461 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.658+0000 7fe7a1200700 1 -- start start 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.658+0000 7fe7a1200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.658+0000 7fe7a1200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c19dc10 con 0x7fe79c1076c0 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.659+0000 7fe799e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.659+0000 7fe799e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33600/0 (socket says 172.21.15.72:33600) 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.659+0000 7fe799e00700 1 -- 172.21.15.72:0/3243995817 learned_addr learned my addr 172.21.15.72:0/3243995817 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.659+0000 7fe799e00700 1 -- 172.21.15.72:0/3243995817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c009550 con 0x7fe79c1076c0 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.659+0000 7fe799e00700 1 --2- 172.21.15.72:0/3243995817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011780 tx=0x7fe78c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe78c003af0 con 0x7fe79c1076c0 2024-03-11T19:10:28.462 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe78c003c50 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c19de10 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe78c0197e0 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c19e2b0 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe78c025460 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.660+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe78c00bb20 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.661+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c197a80 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.664+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe78c02a060 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.740+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe79c02d7c0 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.740+0000 7fe792a00700 1 -- 172.21.15.72:0/3243995817 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fe78c019d60 con 0x7fe79c1076c0 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 msgr2=0x7fe79c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 --2- 172.21.15.72:0/3243995817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011780 tx=0x7fe78c011b40 comp rx=0 tx=0).stop 2024-03-11T19:10:28.463 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 shutdown_connections 2024-03-11T19:10:28.464 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 --2- 172.21.15.72:0/3243995817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe79c1076c0 0x7fe79c19d6d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:28.464 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 >> 172.21.15.72:0/3243995817 conn(0x7fe79c102d00 msgr2=0x7fe79c194bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:28.464 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 shutdown_connections 2024-03-11T19:10:28.464 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:27.742+0000 7fe7a1200700 1 -- 172.21.15.72:0/3243995817 wait complete. 2024-03-11T19:10:28.464 INFO:teuthology.orchestra.run.smithi072.stdout:Restarting the monitor... 2024-03-11T19:10:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:28 smithi072 systemd[1]: Stopping Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:10:29.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a[31317]: 2024-03-11T19:10:28.934+0000 7f66c5a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:10:29.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a[31317]: 2024-03-11T19:10:28.934+0000 7f66c5a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-11T19:10:29.282 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:28 smithi072 podman[31574]: 2024-03-11 19:10:28.987227202 +0000 UTC m=+0.180555621 container stop 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-11T19:10:29.282 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:29 smithi072 podman[31574]: 2024-03-11 19:10:29.016550286 +0000 UTC m=+0.209878666 container died 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T19:10:29.553 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:29 smithi072 podman[31574]: 2024-03-11 19:10:29.38201013 +0000 UTC m=+0.575338543 container cleanup 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:10:29.553 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:29 smithi072 bash[31574]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a 2024-03-11T19:10:29.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:29 smithi072 podman[31588]: 2024-03-11 19:10:29.550281637 +0000 UTC m=+0.540666420 container remove 9844cb517e64307f5820c761d2c84609f21c9eeeec67fc80cd23785d1bfe12ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-11T19:10:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 systemd[1]: ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-11T19:10:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 systemd[1]: Stopped Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:10:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 systemd[1]: ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service: Consumed 1.912s CPU time. 2024-03-11T19:10:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 systemd[1]: Starting Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:10:30.695 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 podman[31677]: 2024-03-11 19:10:30.426980342 +0000 UTC m=+0.128736565 container create d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-11T19:10:30.695 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 podman[31677]: 2024-03-11 19:10:30.334661735 +0000 UTC m=+0.036417921 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:10:30.695 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 podman[31677]: 2024-03-11 19:10:30.69201392 +0000 UTC m=+0.393770149 container init d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True) 2024-03-11T19:10:30.794 INFO:teuthology.orchestra.run.smithi072.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 podman[31677]: 2024-03-11 19:10:30.702411956 +0000 UTC m=+0.404168163 container start d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307) 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: set uid:gid to 167:167 (ceph:ceph) 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: pidfile_write: ignore empty --pid-file 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: load: jerasure load: lrc 2024-03-11T19:10:30.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: RocksDB version: 7.9.2 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Git sha 0 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: DB SUMMARY 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: DB Session ID: LM8M03TZB2KFH0ER1AO7 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: CURRENT file: CURRENT 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: IDENTITY file: IDENTITY 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100868 ; 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.error_if_exists: 0 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.create_if_missing: 0 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.paranoid_checks: 1 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-11T19:10:30.947 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.env: 0x55f28463cae0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.fs: PosixFileSystem 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.info_log: 0x55f286ae7900 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_file_opening_threads: 16 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.statistics: (nil) 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.use_fsync: 0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_log_file_size: 0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.keep_log_file_num: 1000 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.recycle_log_file_num: 0 2024-03-11T19:10:30.948 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_fallocate: 1 2024-03-11T19:10:30.950 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_mmap_reads: 0 2024-03-11T19:10:30.950 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_mmap_writes: 0 2024-03-11T19:10:30.950 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.use_direct_reads: 0 2024-03-11T19:10:30.950 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.create_missing_column_families: 0 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.db_log_dir: 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.wal_dir: 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-11T19:10:30.951 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.advise_random_on_open: 1 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.db_write_buffer_size: 0 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.write_buffer_manager: 0x55f286abb400 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.rate_limiter: (nil) 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.wal_recovery_mode: 2 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enable_thread_tracking: 0 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enable_pipelined_write: 0 2024-03-11T19:10:30.952 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.unordered_write: 0 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.row_cache: None 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.wal_filter: None 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_ingest_behind: 0 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.two_write_queues: 0 2024-03-11T19:10:30.953 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.manual_wal_flush: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.wal_compression: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.atomic_flush: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.log_readahead_size: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.best_efforts_recovery: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.allow_data_in_errors: 0 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.db_host_id: __hostname__ 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-11T19:10:30.954 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_background_jobs: 2 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_background_compactions: -1 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_subcompactions: 1 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_total_wal_size: 0 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_open_files: -1 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bytes_per_sync: 0 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-11T19:10:30.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_readahead_size: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_background_flushes: -1 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Compression algorithms supported: 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kZSTD supported: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kXpressCompression supported: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kLZ4HCCompression supported: 1 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kZlibCompression supported: 1 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kSnappyCompression supported: 1 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kLZ4Compression supported: 1 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: kBZip2Compression supported: 0 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-11T19:10:30.956 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.merge_operator: 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_filter: None 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_filter_factory: None 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.sst_partitioner_factory: None 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f286ae7a00) 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks: 1 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_top_level_index_and_filter: 1 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_type: 0 2024-03-11T19:10:30.957 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_index_type: 0 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_shortening: 1 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: checksum: 4 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: no_block_cache: 0 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache: 0x55f286aeb090 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_name: BinnedLRUCache 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_options: 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: capacity : 536870912 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_shard_bits : 4 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: strict_capacity_limit : 0 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: high_pri_pool_ratio: 0.000 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_compressed: (nil) 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: persistent_cache: (nil) 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size: 4096 2024-03-11T19:10:30.958 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size_deviation: 10 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_restart_interval: 16 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_block_restart_interval: 1 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: metadata_block_size: 4096 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: partition_filters: 0 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: use_delta_encoding: 1 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: filter_policy: bloomfilter 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: whole_key_filtering: 1 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: verify_compression: 0 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: read_amp_bytes_per_bit: 0 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: format_version: 5 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_index_compression: 1 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_align: 0 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: max_auto_readahead_size: 262144 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: prepopulate_block_cache: 0 2024-03-11T19:10:30.959 INFO:journalctl@ceph.mon.a.smithi072.stdout: initial_auto_readahead_size: 8192 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_file_reads_for_auto_readahead: 2 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.write_buffer_size: 33554432 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_write_buffer_number: 2 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression: NoCompression 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression: Disabled 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.prefix_extractor: nullptr 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.num_levels: 7 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-11T19:10:30.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.level: 32767 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.strategy: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.enabled: false 2024-03-11T19:10:30.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.target_file_size_base: 67108864 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-11T19:10:30.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.arena_block_size: 1048576 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.disable_auto_compactions: 0 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-11T19:10:30.963 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.table_properties_collectors: 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.inplace_update_support: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.bloom_locality: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.max_successive_merges: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-11T19:10:30.964 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.paranoid_file_checks: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.force_consistency_checks: 1 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.report_bg_io_stats: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.ttl: 2592000 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enable_blob_files: false 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.min_blob_size: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_file_size: 268435456 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-11T19:10:30.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.blob_file_starting_level: 0 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3be993c0-72a0-40ea-9708-3b297b59e28e 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184230758209, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184230760022, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95949, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93938, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14045, "raw_average_key_size": 50, "raw_value_size": 86304, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710184230, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3be993c0-72a0-40ea-9708-3b297b59e28e", "db_session_id": "LM8M03TZB2KFH0ER1AO7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184230760269, "job": 1, "event": "recovery_finished"} 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f286bdc000 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: DB pointer 0x55f286bc6000 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-11T19:10:30.966 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** DB Stats ** 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: L0 2/0 95.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:10:30.967 INFO:journalctl@ceph.mon.a.smithi072.stdout: Sum 2/0 95.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 79.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-11T19:10:30.968 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative compaction: 0.00 GB write, 14.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval compaction: 0.00 GB write, 14.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache BinnedLRUCache@0x55f286aeb090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: starting mon.a rank 0 at public addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] at bind addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???) e1 preinit fsid d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).mds e1 new map 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).mds e1 print_map 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: e1 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:10:30.969 INFO:journalctl@ceph.mon.a.smithi072.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout: legacy client fscid: -1 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout: No filesystems configured 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 bash[31677]: d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 systemd[1]: Started Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: monmap e1: 1 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: fsmap 2024-03-11T19:10:30.970 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: osdmap e1: 0 total, 0 up, 0 in 2024-03-11T19:10:30.971 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:30 smithi072 ceph-mon[31695]: mgrmap e1: no daemons active 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.429+0000 7f29ec600700 1 Processor -- start 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.429+0000 7f29ec600700 1 -- start start 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.430+0000 7f29ec600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.430+0000 7f29ec600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e407b8e0 con 0x7f29e407ce60 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.430+0000 7f29e9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.430+0000 7f29e9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33614/0 (socket says 172.21.15.72:33614) 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.430+0000 7f29e9c00700 1 -- 172.21.15.72:0/3095681963 learned_addr learned my addr 172.21.15.72:0/3095681963 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.431+0000 7f29e9c00700 1 -- 172.21.15.72:0/3095681963 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e407ba20 con 0x7f29e407ce60 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.431+0000 7f29e9c00700 1 --2- 172.21.15.72:0/3095681963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098a0 tx=0x7f29d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8d4fe3c6bc286a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29e3e00700 1 -- 172.21.15.72:0/3095681963 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29d8004160 con 0x7f29e407ce60 2024-03-11T19:10:32.120 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29e3e00700 1 -- 172.21.15.72:0/3095681963 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29d80042c0 con 0x7f29e407ce60 2024-03-11T19:10:32.121 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29e3e00700 1 -- 172.21.15.72:0/3095681963 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29d8011ac0 con 0x7f29e407ce60 2024-03-11T19:10:32.121 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 -- 172.21.15.72:0/3095681963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 msgr2=0x7f29e407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:32.121 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 --2- 172.21.15.72:0/3095681963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f29d80098a0 tx=0x7f29d8009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:32.121 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 -- 172.21.15.72:0/3095681963 shutdown_connections 2024-03-11T19:10:32.121 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 --2- 172.21.15.72:0/3095681963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e407b310 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 -- 172.21.15.72:0/3095681963 >> 172.21.15.72:0/3095681963 conn(0x7f29e4102d30 msgr2=0x7f29e4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.432+0000 7f29ec600700 1 -- 172.21.15.72:0/3095681963 shutdown_connections 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.433+0000 7f29ec600700 1 -- 172.21.15.72:0/3095681963 wait complete. 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.433+0000 7f29ec600700 1 Processor -- start 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29ec600700 1 -- start start 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29ec600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:32.124 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29ec600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419dbe0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29e9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29e9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33630/0 (socket says 172.21.15.72:33630) 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.434+0000 7f29e9c00700 1 -- 172.21.15.72:0/3863649421 learned_addr learned my addr 172.21.15.72:0/3863649421 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29e9c00700 1 -- 172.21.15.72:0/3863649421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8009550 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29e9c00700 1 --2- 172.21.15.72:0/3863649421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009fb0 tx=0x7f29d8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29d80038b0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29d8003a10 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e419dde0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29d80195e0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.435+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e419e280 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.436+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f29d8019c50 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.436+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f29d80249e0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.436+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e410daa0 con 0x7f29e407ce60 2024-03-11T19:10:32.125 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.440+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f29d8003b80 con 0x7f29e407ce60 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.515+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f29e410dc80 con 0x7f29e407ce60 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.521+0000 7f29e2000700 1 -- 172.21.15.72:0/3863649421 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f29d801aa50 con 0x7f29e407ce60 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 msgr2=0x7f29e419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 --2- 172.21.15.72:0/3863649421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009fb0 tx=0x7f29d8011b20 comp rx=0 tx=0).stop 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 shutdown_connections 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 --2- 172.21.15.72:0/3863649421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e407ce60 0x7f29e419d6a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 >> 172.21.15.72:0/3863649421 conn(0x7f29e4102d30 msgr2=0x7f29e41038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 shutdown_connections 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:31.522+0000 7f29ec600700 1 -- 172.21.15.72:0/3863649421 wait complete. 2024-03-11T19:10:32.126 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-11T19:10:32.149 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:10:32.149 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mgr... 2024-03-11T19:10:32.151 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-11T19:10:32.152 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-11T19:10:32.584 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x 2024-03-11T19:10:32.584 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x.service: Unit ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x.service not loaded. 2024-03-11T19:10:32.840 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715.target.wants/ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-d6472314-dfda-11ee-95c8-87774f69a715@.service. 2024-03-11T19:10:32.875 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:32 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3863649421' entity='client.admin' 2024-03-11T19:10:33.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:32 smithi072 systemd[1]: Starting Ceph mgr.x for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:10:33.518 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 podman[31904]: 2024-03-11 19:10:33.21929845 +0000 UTC m=+0.129204346 container create 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-11T19:10:33.518 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 podman[31904]: 2024-03-11 19:10:33.128713855 +0000 UTC m=+0.038619704 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:10:33.518 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 podman[31904]: 2024-03-11 19:10:33.441060095 +0000 UTC m=+0.350965970 container init 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-11T19:10:33.518 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 podman[31904]: 2024-03-11 19:10:33.451396198 +0000 UTC m=+0.361302038 container start 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397) 2024-03-11T19:10:33.518 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 bash[31904]: 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 2024-03-11T19:10:33.546 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:10:33.546 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-03-11T19:10:33.561 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:10:33.561 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:10:33.561 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-03-11T19:10:33.561 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:10:33.579 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:10:33.579 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr to start... 2024-03-11T19:10:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr... 2024-03-11T19:10:33.870 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:33 smithi072 systemd[1]: Started Ceph mgr.x for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:10:34.480 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:34.218+0000 7f265f928200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:10:34.480 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:34.343+0000 7f265f928200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:10:34.851 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:34 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1297125655' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:10:35.153 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:35.154 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:10:35.155 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:35.156 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:10:35.156 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:10:35.156 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:10:35.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:35.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:10:35.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:35.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:10:24.350547+0000", 2024-03-11T19:10:35.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:35.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.370+0000 7f88c5400700 1 Processor -- start 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.370+0000 7f88c5400700 1 -- start start 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.371+0000 7f88c5400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.371+0000 7f88c5400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c007b900 con 0x7f88c007ce80 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.371+0000 7f88be800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.371+0000 7f88be800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59624/0 (socket says 172.21.15.72:59624) 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.371+0000 7f88be800700 1 -- 172.21.15.72:0/853069996 learned_addr learned my addr 172.21.15.72:0/853069996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.372+0000 7f88be800700 1 -- 172.21.15.72:0/853069996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c007ba40 con 0x7f88c007ce80 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.372+0000 7f88be800700 1 --2- 172.21.15.72:0/853069996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88b00098a0 tx=0x7f88b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0b24c7230b7b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.372+0000 7f88bd400700 1 -- 172.21.15.72:0/853069996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b0004160 con 0x7f88c007ce80 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.372+0000 7f88bd400700 1 -- 172.21.15.72:0/853069996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b00042c0 con 0x7f88c007ce80 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.372+0000 7f88bd400700 1 -- 172.21.15.72:0/853069996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b0011ac0 con 0x7f88c007ce80 2024-03-11T19:10:35.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 -- 172.21.15.72:0/853069996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 msgr2=0x7f88c007b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 --2- 172.21.15.72:0/853069996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88b00098a0 tx=0x7f88b0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 -- 172.21.15.72:0/853069996 shutdown_connections 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 --2- 172.21.15.72:0/853069996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c007b330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 -- 172.21.15.72:0/853069996 >> 172.21.15.72:0/853069996 conn(0x7f88c0102cd0 msgr2=0x7f88c01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.373+0000 7f88c5400700 1 -- 172.21.15.72:0/853069996 shutdown_connections 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.374+0000 7f88c5400700 1 -- 172.21.15.72:0/853069996 wait complete. 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88c5400700 1 Processor -- start 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88c5400700 1 -- start start 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88c5400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88c5400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019dbe0 con 0x7f88c007ce80 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88be800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88be800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59630/0 (socket says 172.21.15.72:59630) 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.375+0000 7f88be800700 1 -- 172.21.15.72:0/1297125655 learned_addr learned my addr 172.21.15.72:0/1297125655 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88be800700 1 -- 172.21.15.72:0/1297125655 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0009550 con 0x7f88c007ce80 2024-03-11T19:10:35.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88be800700 1 --2- 172.21.15.72:0/1297125655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88b0011a70 tx=0x7f88b0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b00039c0 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c019dde0 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b0003b20 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b0024610 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.376+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c019e280 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.377+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f88b0024c00 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.377+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f88b0019920 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.378+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c010da60 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.380+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f88b001aa10 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.459+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f88c01a0df0 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.459+0000 7f88b7400700 1 -- 172.21.15.72:0/1297125655 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f88b001abb0 con 0x7f88c007ce80 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.460+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 msgr2=0x7f88c019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.460+0000 7f88c5400700 1 --2- 172.21.15.72:0/1297125655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88b0011a70 tx=0x7f88b0011b50 comp rx=0 tx=0).stop 2024-03-11T19:10:35.169 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.461+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 shutdown_connections 2024-03-11T19:10:35.170 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.461+0000 7f88c5400700 1 --2- 172.21.15.72:0/1297125655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88c007ce80 0x7f88c019d6a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:35.170 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.461+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 >> 172.21.15.72:0/1297125655 conn(0x7f88c0102cd0 msgr2=0x7f88c01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:35.170 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.461+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 shutdown_connections 2024-03-11T19:10:35.170 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:34.461+0000 7f88c5400700 1 -- 172.21.15.72:0/1297125655 wait complete. 2024-03-11T19:10:35.170 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (1/15)... 2024-03-11T19:10:35.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:35.157+0000 7f265f928200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:10:35.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:35.278+0000 7f265f928200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:10:35.929 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:35.454+0000 7f265f928200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:10:35.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:35.576+0000 7f265f928200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:10:35.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:35.683+0000 7f265f928200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:10:36.373 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:36.005+0000 7f265f928200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:10:36.373 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:36.141+0000 7f265f928200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:10:36.374 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:36.249+0000 7f265f928200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:10:36.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:36.369+0000 7f265f928200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:10:36.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:36.581+0000 7f265f928200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:10:38.086 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:37 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1683115686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:10:38.087 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:37.911+0000 7f265f928200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:10:38.087 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:38.082+0000 7f265f928200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:10:38.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:10:38.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:10:38.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:10:24.350547+0000", 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.702+0000 7fc47b800700 1 Processor -- start 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.702+0000 7fc47b800700 1 -- start start 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.703+0000 7fc47b800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.703+0000 7fc47b800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4741080b0 con 0x7fc4741076c0 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.703+0000 7fc478e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.703+0000 7fc478e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59634/0 (socket says 172.21.15.72:59634) 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.703+0000 7fc478e00700 1 -- 172.21.15.72:0/767971964 learned_addr learned my addr 172.21.15.72:0/767971964 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:38.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.704+0000 7fc478e00700 1 -- 172.21.15.72:0/767971964 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474108870 con 0x7fc4741076c0 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.704+0000 7fc478e00700 1 --2- 172.21.15.72:0/767971964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4680098a0 tx=0x7fc468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab473648f3832c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.704+0000 7fc473400700 1 -- 172.21.15.72:0/767971964 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc468004160 con 0x7fc4741076c0 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.704+0000 7fc473400700 1 -- 172.21.15.72:0/767971964 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4680042c0 con 0x7fc4741076c0 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.704+0000 7fc473400700 1 -- 172.21.15.72:0/767971964 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc468011ac0 con 0x7fc4741076c0 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 -- 172.21.15.72:0/767971964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 msgr2=0x7fc474107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 --2- 172.21.15.72:0/767971964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4680098a0 tx=0x7fc468009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 -- 172.21.15.72:0/767971964 shutdown_connections 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 --2- 172.21.15.72:0/767971964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc474107ae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 -- 172.21.15.72:0/767971964 >> 172.21.15.72:0/767971964 conn(0x7fc474102d00 msgr2=0x7fc474105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 -- 172.21.15.72:0/767971964 shutdown_connections 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.705+0000 7fc47b800700 1 -- 172.21.15.72:0/767971964 wait complete. 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.707+0000 7fc47b800700 1 Processor -- start 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.707+0000 7fc47b800700 1 -- start start 2024-03-11T19:10:38.427 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.707+0000 7fc47b800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.707+0000 7fc47b800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47407d0c0 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.707+0000 7fc478e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc478e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59648/0 (socket says 172.21.15.72:59648) 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc478e00700 1 -- 172.21.15.72:0/1683115686 learned_addr learned my addr 172.21.15.72:0/1683115686 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc478e00700 1 -- 172.21.15.72:0/1683115686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009550 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc478e00700 1 --2- 172.21.15.72:0/1683115686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc468011780 tx=0x7fc468011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc468003af0 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc468003c50 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.708+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47407b2b0 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.709+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4680197e0 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.709+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47407b750 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.709+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc468025460 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.709+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc46800bb30 con 0x7fc4741076c0 2024-03-11T19:10:38.428 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.710+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47410baa0 con 0x7fc4741076c0 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.712+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc46802a060 con 0x7fc4741076c0 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.790+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc47410bcb0 con 0x7fc4741076c0 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.791+0000 7fc471600700 1 -- 172.21.15.72:0/1683115686 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc468019da0 con 0x7fc4741076c0 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.792+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 msgr2=0x7fc47407cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.792+0000 7fc47b800700 1 --2- 172.21.15.72:0/1683115686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc468011780 tx=0x7fc468011b40 comp rx=0 tx=0).stop 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.792+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 shutdown_connections 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.792+0000 7fc47b800700 1 --2- 172.21.15.72:0/1683115686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4741076c0 0x7fc47407cb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.792+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 >> 172.21.15.72:0/1683115686 conn(0x7fc474102d00 msgr2=0x7fc4741038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.793+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 shutdown_connections 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:37.793+0000 7fc47b800700 1 -- 172.21.15.72:0/1683115686 wait complete. 2024-03-11T19:10:38.429 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (2/15)... 2024-03-11T19:10:38.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:38.498+0000 7f265f928200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:10:38.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:38.614+0000 7f265f928200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:10:39.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:38.865+0000 7f265f928200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:10:39.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:39.103+0000 7f265f928200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:10:39.782 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:39.465+0000 7f265f928200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:10:40.080 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:39.777+0000 7f265f928200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:10:40.080 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:40.075+0000 7f265f928200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:10:41.120 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:41 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:41.094+0000 7f265f928200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:10:41.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:41 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4114036474' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.791 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:10:41.792 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:10:41.793 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:10:24.350547+0000", 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:41.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.008+0000 7f28cb000700 1 Processor -- start 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.009+0000 7f28cb000700 1 -- start start 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.009+0000 7f28cb000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.009+0000 7f28cb000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c407b830 con 0x7f28c407ce40 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.009+0000 7f28c9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59656/0 (socket says 172.21.15.72:59656) 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c9c00700 1 -- 172.21.15.72:0/3027281511 learned_addr learned my addr 172.21.15.72:0/3027281511 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c9c00700 1 -- 172.21.15.72:0/3027281511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c407b970 con 0x7f28c407ce40 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c9c00700 1 --2- 172.21.15.72:0/3027281511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f28b80098a0 tx=0x7f28b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a04321c4b0e44f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c3e00700 1 -- 172.21.15.72:0/3027281511 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28b8004160 con 0x7f28c407ce40 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c3e00700 1 -- 172.21.15.72:0/3027281511 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28b80042c0 con 0x7f28c407ce40 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.010+0000 7f28c3e00700 1 -- 172.21.15.72:0/3027281511 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28b8011ac0 con 0x7f28c407ce40 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 -- 172.21.15.72:0/3027281511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 msgr2=0x7f28c407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:41.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 --2- 172.21.15.72:0/3027281511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f28b80098a0 tx=0x7f28b8009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 -- 172.21.15.72:0/3027281511 shutdown_connections 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 --2- 172.21.15.72:0/3027281511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c407b2f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 -- 172.21.15.72:0/3027281511 >> 172.21.15.72:0/3027281511 conn(0x7f28c4102a80 msgr2=0x7f28c4104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 -- 172.21.15.72:0/3027281511 shutdown_connections 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.011+0000 7f28cb000700 1 -- 172.21.15.72:0/3027281511 wait complete. 2024-03-11T19:10:41.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.012+0000 7f28cb000700 1 Processor -- start 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.012+0000 7f28cb000700 1 -- start start 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28cb000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28cb000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c419da10 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28c9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28c9c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59670/0 (socket says 172.21.15.72:59670) 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28c9c00700 1 -- 172.21.15.72:0/4114036474 learned_addr learned my addr 172.21.15.72:0/4114036474 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.013+0000 7f28c9c00700 1 -- 172.21.15.72:0/4114036474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009550 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c9c00700 1 --2- 172.21.15.72:0/4114036474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011a70 tx=0x7f28b8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28b80039c0 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28b8003b20 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c419dc10 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28b8024610 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c419e0b0 con 0x7f28c407ce40 2024-03-11T19:10:41.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f28b8024bb0 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.014+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f28b80198d0 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.015+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c410d830 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.018+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f28b801aa10 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.096+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f28c41a0c20 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.097+0000 7f28c2000700 1 -- 172.21.15.72:0/4114036474 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f28b801abb0 con 0x7f28c407ce40 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 msgr2=0x7f28c419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 --2- 172.21.15.72:0/4114036474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011a70 tx=0x7f28b8011b50 comp rx=0 tx=0).stop 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 shutdown_connections 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 --2- 172.21.15.72:0/4114036474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28c407ce40 0x7f28c419d4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 >> 172.21.15.72:0/4114036474 conn(0x7f28c4102a80 msgr2=0x7f28c41036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 shutdown_connections 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:41.098+0000 7f28cb000700 1 -- 172.21.15.72:0/4114036474 wait complete. 2024-03-11T19:10:41.798 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (3/15)... 2024-03-11T19:10:42.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:41 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:41.809+0000 7f265f928200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:10:42.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:41 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:41.935+0000 7f265f928200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:42.209+0000 7f265f928200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: Activating manager daemon x 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: mgrmap e2: x(active, starting, since 0.0106388s) 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: Manager daemon x is now available 2024-03-11T19:10:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:10:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:10:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:42 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' 2024-03-11T19:10:43.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:43 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' 2024-03-11T19:10:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:43 smithi072 ceph-mon[31695]: from='mgr.14100 172.21.15.72:0/3134538351' entity='mgr.x' 2024-03-11T19:10:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:43 smithi072 ceph-mon[31695]: mgrmap e3: x(active, since 1.02183s) 2024-03-11T19:10:45.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:45 smithi072 ceph-mon[31695]: mgrmap e4: x(active, since 2s) 2024-03-11T19:10:45.561 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:45 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1546134429' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:10:45.580 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:10:45.581 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:10:45.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:10:45.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:10:24.350547+0000", 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:45.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71bae00700 1 Processor -- start 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71bae00700 1 -- start start 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71bae00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71bae00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b407ae80 con 0x7f71b4105f30 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52492/0 (socket says 172.21.15.72:52492) 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71b3e00700 1 -- 172.21.15.72:0/1193891339 learned_addr learned my addr 172.21.15.72:0/1193891339 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.418+0000 7f71b3e00700 1 -- 172.21.15.72:0/1193891339 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4106890 con 0x7f71b4105f30 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.419+0000 7f71b3e00700 1 --2- 172.21.15.72:0/1193891339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098a0 tx=0x7f71a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717608ae329148ec server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.419+0000 7f71b2a00700 1 -- 172.21.15.72:0/1193891339 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a4004160 con 0x7f71b4105f30 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.419+0000 7f71b2a00700 1 -- 172.21.15.72:0/1193891339 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71a40042c0 con 0x7f71b4105f30 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.419+0000 7f71b2a00700 1 -- 172.21.15.72:0/1193891339 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a4011ac0 con 0x7f71b4105f30 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 -- 172.21.15.72:0/1193891339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 msgr2=0x7f71b4106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 --2- 172.21.15.72:0/1193891339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098a0 tx=0x7f71a4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 -- 172.21.15.72:0/1193891339 shutdown_connections 2024-03-11T19:10:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 --2- 172.21.15.72:0/1193891339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b4106350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 -- 172.21.15.72:0/1193891339 >> 172.21.15.72:0/1193891339 conn(0x7f71b4101b90 msgr2=0x7f71b4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 -- 172.21.15.72:0/1193891339 shutdown_connections 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.420+0000 7f71bae00700 1 -- 172.21.15.72:0/1193891339 wait complete. 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.421+0000 7f71bae00700 1 Processor -- start 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.421+0000 7f71bae00700 1 -- start start 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.421+0000 7f71bae00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.421+0000 7f71bae00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419dbd0 con 0x7f71b4105f30 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52504/0 (socket says 172.21.15.72:52504) 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b3e00700 1 -- 172.21.15.72:0/1546134429 learned_addr learned my addr 172.21.15.72:0/1546134429 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b3e00700 1 -- 172.21.15.72:0/1546134429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009550 con 0x7f71b4105f30 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b3e00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011a70 tx=0x7f71a4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.422+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a40039c0 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.423+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71a4003b20 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.423+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a4024610 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.423+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b419ddd0 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.423+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b419e270 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.424+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f71a401aa10 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.424+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b4197300 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.424+0000 7f71b0c00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 0x7f719803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.424+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f71a404cbb0 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.425+0000 7f71b3400700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 0x7f719803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.425+0000 7f71b3400700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 0x7f719803b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.431+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71a401a430 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.734+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f71b4107760 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.735+0000 7f71b0c00700 1 -- 172.21.15.72:0/1546134429 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f71a4018be0 con 0x7f71b4105f30 2024-03-11T19:10:45.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.737+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 msgr2=0x7f719803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.737+0000 7f71bae00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 0x7f719803b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0006e40 comp rx=0 tx=0).stop 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 msgr2=0x7f71b419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011a70 tx=0x7f71a4011b50 comp rx=0 tx=0).stop 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 shutdown_connections 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f7198038cb0 0x7f719803b170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 --2- 172.21.15.72:0/1546134429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71b4105f30 0x7f71b419d690 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 >> 172.21.15.72:0/1546134429 conn(0x7f71b4101b90 msgr2=0x7f71b4194440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 shutdown_connections 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:44.738+0000 7f71bae00700 1 -- 172.21.15.72:0/1546134429 wait complete. 2024-03-11T19:10:45.588 INFO:teuthology.orchestra.run.smithi072.stdout:mgr is available 2024-03-11T19:10:46.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1664016453' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-11T19:10:46.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:46.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:10:46.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:46.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:10:46.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:10:46.940 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.106+0000 7ff7d4400700 1 Processor -- start 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.106+0000 7ff7d4400700 1 -- start start 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.107+0000 7ff7d4400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.107+0000 7ff7d4400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc07b8e0 con 0x7ff7cc07ce60 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.107+0000 7ff7d1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.107+0000 7ff7d1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52508/0 (socket says 172.21.15.72:52508) 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.107+0000 7ff7d1a00700 1 -- 172.21.15.72:0/1442875739 learned_addr learned my addr 172.21.15.72:0/1442875739 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.108+0000 7ff7d1a00700 1 -- 172.21.15.72:0/1442875739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc07ba20 con 0x7ff7cc07ce60 2024-03-11T19:10:46.941 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.108+0000 7ff7d1a00700 1 --2- 172.21.15.72:0/1442875739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009e80 tx=0x7ff7c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c42a56825ea637a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.108+0000 7ff7cbe00700 1 -- 172.21.15.72:0/1442875739 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c000e070 con 0x7ff7cc07ce60 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.108+0000 7ff7cbe00700 1 -- 172.21.15.72:0/1442875739 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7c000b8c0 con 0x7ff7cc07ce60 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.108+0000 7ff7cbe00700 1 -- 172.21.15.72:0/1442875739 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c0004550 con 0x7ff7cc07ce60 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 -- 172.21.15.72:0/1442875739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 msgr2=0x7ff7cc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1442875739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009e80 tx=0x7ff7c000b330 comp rx=0 tx=0).stop 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 -- 172.21.15.72:0/1442875739 shutdown_connections 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1442875739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc07b310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 -- 172.21.15.72:0/1442875739 >> 172.21.15.72:0/1442875739 conn(0x7ff7cc102d00 msgr2=0x7ff7cc105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.109+0000 7ff7d4400700 1 -- 172.21.15.72:0/1442875739 shutdown_connections 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.110+0000 7ff7d4400700 1 -- 172.21.15.72:0/1442875739 wait complete. 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d4400700 1 Processor -- start 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d4400700 1 -- start start 2024-03-11T19:10:46.942 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d4400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d4400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc19db00 con 0x7ff7cc07ce60 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52518/0 (socket says 172.21.15.72:52518) 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.111+0000 7ff7d1a00700 1 -- 172.21.15.72:0/1664016453 learned_addr learned my addr 172.21.15.72:0/1664016453 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7d1a00700 1 -- 172.21.15.72:0/1664016453 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c0009b30 con 0x7ff7cc07ce60 2024-03-11T19:10:46.943 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7d1a00700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009fd0 tx=0x7ff7c0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:46.945 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c000e040 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7c00128f0 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc19dd00 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7c001c770 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.112+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc19e180 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.113+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7ff7c0012a60 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.113+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc10da90 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.114+0000 7ff7ca000700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 0x7ff7b403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.114+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff7c004cc60 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.114+0000 7ff7d1000700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 0x7ff7b403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.115+0000 7ff7d1000700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 0x7ff7b403b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8006fd0 tx=0x7ff7b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.119+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7c0024c40 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.336+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff7cc10dca0 con 0x7ff7cc07ce60 2024-03-11T19:10:46.946 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.338+0000 7ff7ca000700 1 -- 172.21.15.72:0/1664016453 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff7c0017ea0 con 0x7ff7cc07ce60 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 msgr2=0x7ff7b403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 0x7ff7b403b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8006fd0 tx=0x7ff7b8006e40 comp rx=0 tx=0).stop 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 msgr2=0x7ff7cc19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009fd0 tx=0x7ff7c0003d00 comp rx=0 tx=0).stop 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 shutdown_connections 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7ff7b4038cb0 0x7ff7b403b170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 --2- 172.21.15.72:0/1664016453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7cc07ce60 0x7ff7cc19d5c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.341+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 >> 172.21.15.72:0/1664016453 conn(0x7ff7cc102d00 msgr2=0x7ff7cc1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.342+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 shutdown_connections 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:46.342+0000 7ff7d4400700 1 -- 172.21.15.72:0/1664016453 wait complete. 2024-03-11T19:10:46.947 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling cephadm module... 2024-03-11T19:10:48.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:47 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1442640148' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-11T19:10:48.158 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:47 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setuser ceph since I am not root 2024-03-11T19:10:48.158 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:47 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setgroup ceph since I am not root 2024-03-11T19:10:48.531 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.520+0000 7f4b17a00700 1 Processor -- start 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.521+0000 7f4b17a00700 1 -- start start 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.521+0000 7f4b17a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.521+0000 7f4b17a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b101080b0 con 0x7f4b101076c0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.521+0000 7f4b15000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.521+0000 7f4b15000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52524/0 (socket says 172.21.15.72:52524) 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.522+0000 7f4b15000700 1 -- 172.21.15.72:0/3584286952 learned_addr learned my addr 172.21.15.72:0/3584286952 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.522+0000 7f4b15000700 1 -- 172.21.15.72:0/3584286952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10108870 con 0x7f4b101076c0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.522+0000 7f4b15000700 1 --2- 172.21.15.72:0/3584286952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4af40098a0 tx=0x7f4af4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4420419532e3808c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.522+0000 7f4b0fe00700 1 -- 172.21.15.72:0/3584286952 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4af4004160 con 0x7f4b101076c0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.522+0000 7f4b0fe00700 1 -- 172.21.15.72:0/3584286952 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af40042c0 con 0x7f4b101076c0 2024-03-11T19:10:48.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b0fe00700 1 -- 172.21.15.72:0/3584286952 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4af4011ac0 con 0x7f4b101076c0 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b17a00700 1 -- 172.21.15.72:0/3584286952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 msgr2=0x7f4b10107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b17a00700 1 --2- 172.21.15.72:0/3584286952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4af40098a0 tx=0x7f4af4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b17a00700 1 -- 172.21.15.72:0/3584286952 shutdown_connections 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b17a00700 1 --2- 172.21.15.72:0/3584286952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b10107ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.523+0000 7f4b17a00700 1 -- 172.21.15.72:0/3584286952 >> 172.21.15.72:0/3584286952 conn(0x7f4b10102d20 msgr2=0x7f4b10105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.524+0000 7f4b17a00700 1 -- 172.21.15.72:0/3584286952 shutdown_connections 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.524+0000 7f4b17a00700 1 -- 172.21.15.72:0/3584286952 wait complete. 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b17a00700 1 Processor -- start 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b17a00700 1 -- start start 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b17a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b17a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1019dbf0 con 0x7f4b101076c0 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b15000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.525+0000 7f4b15000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52536/0 (socket says 172.21.15.72:52536) 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b15000700 1 -- 172.21.15.72:0/1442640148 learned_addr learned my addr 172.21.15.72:0/1442640148 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:48.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b15000700 1 -- 172.21.15.72:0/1442640148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4009550 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b15000700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4af4011780 tx=0x7f4af4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4af4003af0 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af4003c50 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1019ddf0 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.526+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4af40197e0 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.527+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1019e290 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.528+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f4af401aba0 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.528+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1010baa0 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.528+0000 7f4b0e000700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 0x7f4aec03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.528+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4af404cf50 con 0x7f4b101076c0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.529+0000 7f4b07e00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 0x7f4aec03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.530+0000 7f4b07e00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 0x7f4aec03b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:48.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.534+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4af401a430 con 0x7f4b101076c0 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.810+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f4b1010bcb0 con 0x7f4b101076c0 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.864+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f4af402dbe0 con 0x7f4b101076c0 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b0e000700 1 -- 172.21.15.72:0/1442640148 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f4af402d350 con 0x7f4b101076c0 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 msgr2=0x7f4aec03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 0x7f4aec03b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc006e40 comp rx=0 tx=0).stop 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 msgr2=0x7f4b1019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4af4011780 tx=0x7f4af4011b40 comp rx=0 tx=0).stop 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 shutdown_connections 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f4aec038cb0 0x7f4aec03b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 --2- 172.21.15.72:0/1442640148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b101076c0 0x7f4b1019d6b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.868+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 >> 172.21.15.72:0/1442640148 conn(0x7f4b10102d20 msgr2=0x7f4b10103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.869+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 shutdown_connections 2024-03-11T19:10:48.535 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:47.869+0000 7f4b17a00700 1 -- 172.21.15.72:0/1442640148 wait complete. 2024-03-11T19:10:48.790 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:48 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:48.627+0000 7f2a5b37d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:10:48.790 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:48 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:48.747+0000 7f2a5b37d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:10:49.054 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1442640148' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-11T19:10:49.054 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:48 smithi072 ceph-mon[31695]: mgrmap e5: x(active, since 5s) 2024-03-11T19:10:49.670 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:49.545+0000 7f2a5b37d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:10:49.670 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:49.667+0000 7f2a5b37d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:10:49.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3308686511' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:10:49.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:49.843+0000 7f2a5b37d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:10:50.089 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:50.089 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-11T19:10:50.089 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:10:50.089 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:10:50.089 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.051+0000 7fe633200700 1 Processor -- start 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.051+0000 7fe633200700 1 -- start start 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.051+0000 7fe633200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.051+0000 7fe633200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c07aec0 con 0x7fe62c105ef0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.052+0000 7fe62be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.052+0000 7fe62be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52570/0 (socket says 172.21.15.72:52570) 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.052+0000 7fe62be00700 1 -- 172.21.15.72:0/3834978816 learned_addr learned my addr 172.21.15.72:0/3834978816 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.052+0000 7fe62be00700 1 -- 172.21.15.72:0/3834978816 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c108860 con 0x7fe62c105ef0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.053+0000 7fe62be00700 1 --2- 172.21.15.72:0/3834978816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0098a0 tx=0x7fe61c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6641d796405bc47e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.053+0000 7fe62aa00700 1 -- 172.21.15.72:0/3834978816 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe61c004160 con 0x7fe62c105ef0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.053+0000 7fe62aa00700 1 -- 172.21.15.72:0/3834978816 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe61c0042c0 con 0x7fe62c105ef0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.053+0000 7fe62aa00700 1 -- 172.21.15.72:0/3834978816 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe61c011ac0 con 0x7fe62c105ef0 2024-03-11T19:10:50.090 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 -- 172.21.15.72:0/3834978816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 msgr2=0x7fe62c108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 --2- 172.21.15.72:0/3834978816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0098a0 tx=0x7fe61c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 -- 172.21.15.72:0/3834978816 shutdown_connections 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 --2- 172.21.15.72:0/3834978816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c108320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 -- 172.21.15.72:0/3834978816 >> 172.21.15.72:0/3834978816 conn(0x7fe62c101b90 msgr2=0x7fe62c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 -- 172.21.15.72:0/3834978816 shutdown_connections 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.054+0000 7fe633200700 1 -- 172.21.15.72:0/3834978816 wait complete. 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.055+0000 7fe633200700 1 Processor -- start 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.055+0000 7fe633200700 1 -- start start 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.055+0000 7fe633200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.055+0000 7fe633200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1a1ff0 con 0x7fe62c105ef0 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe62be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe62be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52584/0 (socket says 172.21.15.72:52584) 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe62be00700 1 -- 172.21.15.72:0/3308686511 learned_addr learned my addr 172.21.15.72:0/3308686511 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:50.091 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe62be00700 1 -- 172.21.15.72:0/3308686511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009550 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe62be00700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe61c011a70 tx=0x7fe61c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.056+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe61c0039c0 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.057+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe61c003b20 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.057+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c1a21f0 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.057+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe61c024610 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.057+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c1a2690 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe61c02d3f0 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c005360 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe628c00700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 0x7fe61003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe61c04dc40 con 0x7fe62c105ef0 2024-03-11T19:10:50.092 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe62b400700 1 -- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 msgr2=0x7fe61003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.058+0000 7fe62b400700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 0x7fe61003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.068+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe61c010970 con 0x7fe62c105ef0 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.259+0000 7fe62b400700 1 -- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 msgr2=0x7fe61003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.259+0000 7fe62b400700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 0x7fe61003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.343+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe60c005e20 con 0x7fe62c105ef0 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.343+0000 7fe628c00700 1 -- 172.21.15.72:0/3308686511 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe61c00b3e0 con 0x7fe62c105ef0 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 msgr2=0x7fe61003b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 0x7fe61003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 msgr2=0x7fe62c1a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe61c011a70 tx=0x7fe61c011b50 comp rx=0 tx=0).stop 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 shutdown_connections 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7fe610038d00 0x7fe61003b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 --2- 172.21.15.72:0/3308686511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe62c105ef0 0x7fe62c1a1ab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:50.093 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.346+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 >> 172.21.15.72:0/3308686511 conn(0x7fe62c101b90 msgr2=0x7fe62c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:50.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.347+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 shutdown_connections 2024-03-11T19:10:50.094 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:49.347+0000 7fe633200700 1 -- 172.21.15.72:0/3308686511 wait complete. 2024-03-11T19:10:50.094 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-03-11T19:10:50.094 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 5... 2024-03-11T19:10:50.231 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:49.968+0000 7f2a5b37d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:10:50.231 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:50.077+0000 7f2a5b37d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:10:50.512 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:50.422+0000 7f2a5b37d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:10:50.807 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:50.563+0000 7f2a5b37d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:10:50.807 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:50.681+0000 7f2a5b37d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:10:50.807 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:50.803+0000 7f2a5b37d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:10:51.179 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:51 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:51.018+0000 7f2a5b37d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:10:52.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:52.263+0000 7f2a5b37d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:10:52.840 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:52.429+0000 7f2a5b37d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:10:53.179 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:52.836+0000 7f2a5b37d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:10:53.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:52.953+0000 7f2a5b37d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:10:53.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:53 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:53.204+0000 7f2a5b37d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:10:53.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:53 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:53.442+0000 7f2a5b37d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:10:53.929 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:53 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:53.804+0000 7f2a5b37d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:10:54.415 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:54 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:54.116+0000 7f2a5b37d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:10:54.679 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:54 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:54.412+0000 7f2a5b37d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:10:55.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:55 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:55.462+0000 7f2a5b37d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:10:56.286 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:56.155+0000 7f2a5b37d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:10:56.287 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:56.282+0000 7f2a5b37d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:10:56.558+0000 7f2a5b37d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: Active manager daemon x restarted 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: Activating manager daemon x 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: osdmap e2: 0 total, 0 up, 0 in 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: mgrmap e6: x(active, starting, since 0.0076902s) 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:10:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: Manager daemon x is now available 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:10:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:56 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:10:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: Found migration_current of "None". Setting to last migration. 2024-03-11T19:10:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:10:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:10:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:10:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:10:57.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:10:57.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:10:57.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:10:57.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:10:57.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:57 smithi072 ceph-mon[31695]: mgrmap e7: x(active, since 1.01658s) 2024-03-11T19:10:58.210 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:10:58.218 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-11T19:10:58.218 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:10:58.218 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.661+0000 7f50f3800700 1 Processor -- start 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.661+0000 7f50f3800700 1 -- start start 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.662+0000 7f50f3800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.662+0000 7f50f3800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec07b8e0 con 0x7f50ec07ce60 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.662+0000 7f50f0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.662+0000 7f50f0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52592/0 (socket says 172.21.15.72:52592) 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.662+0000 7f50f0e00700 1 -- 172.21.15.72:0/215078092 learned_addr learned my addr 172.21.15.72:0/215078092 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.663+0000 7f50f0e00700 1 -- 172.21.15.72:0/215078092 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec07ba20 con 0x7f50ec07ce60 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.663+0000 7f50f0e00700 1 --2- 172.21.15.72:0/215078092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098a0 tx=0x7f50e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3eeda569d024c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.663+0000 7f50eb400700 1 -- 172.21.15.72:0/215078092 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50e0004160 con 0x7f50ec07ce60 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.663+0000 7f50eb400700 1 -- 172.21.15.72:0/215078092 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50e00042c0 con 0x7f50ec07ce60 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.663+0000 7f50eb400700 1 -- 172.21.15.72:0/215078092 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50e0011ac0 con 0x7f50ec07ce60 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 -- 172.21.15.72:0/215078092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 msgr2=0x7f50ec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:58.219 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 --2- 172.21.15.72:0/215078092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f50e00098a0 tx=0x7f50e0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 -- 172.21.15.72:0/215078092 shutdown_connections 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 --2- 172.21.15.72:0/215078092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec07b310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 -- 172.21.15.72:0/215078092 >> 172.21.15.72:0/215078092 conn(0x7f50ec102cd0 msgr2=0x7f50ec1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 -- 172.21.15.72:0/215078092 shutdown_connections 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.664+0000 7f50f3800700 1 -- 172.21.15.72:0/215078092 wait complete. 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.666+0000 7f50f3800700 1 Processor -- start 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.666+0000 7f50f3800700 1 -- start start 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.666+0000 7f50f3800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.666+0000 7f50f3800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec19dbe0 con 0x7f50ec07ce60 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.667+0000 7f50f0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.667+0000 7f50f0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52606/0 (socket says 172.21.15.72:52606) 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.667+0000 7f50f0e00700 1 -- 172.21.15.72:0/386297550 learned_addr learned my addr 172.21.15.72:0/386297550 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:58.220 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.667+0000 7f50f0e00700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009550 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.667+0000 7f50f0e00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50e0011a70 tx=0x7f50e0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.668+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50e00039c0 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.668+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50e0003b20 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.668+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec19dde0 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.668+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50e0024610 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.668+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec19e280 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.669+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f50e002d3f0 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.669+0000 7f50e9600700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.669+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50d403b880 con 0x7f50d4038cb0 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.669+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f50e004cc50 con 0x7f50ec07ce60 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.670+0000 7f50ebe00700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.670+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:10:58.221 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.871+0000 7f50ebe00700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:50.871+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:51.272+0000 7f50ebe00700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:51.272+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:52.073+0000 7f50ebe00700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:52.073+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:53.675+0000 7f50ebe00700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1095172224 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:53.675+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:56.567+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7f50e0021d30 con 0x7f50ec07ce60 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:56.567+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 msgr2=0x7f50d403b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:56.567+0000 7f50e9600700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.571+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f50e004e350 con 0x7f50ec07ce60 2024-03-11T19:10:58.222 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.572+0000 7f50e9600700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 0x7f50d403e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.572+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50d403b880 con 0x7f50d403c5a0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.575+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 0x7f50d403e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.577+0000 7f50ebe00700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 0x7f50d403e990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d8003a10 tx=0x7f50d8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.579+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f50d403b880 con 0x7f50d403c5a0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.588+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f50ec10da80 con 0x7f50d403c5a0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.589+0000 7f50e9600700 1 -- 172.21.15.72:0/386297550 <== mgr.14120 v2:172.21.15.72:6800/2769969675 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f50ec10da80 con 0x7f50d403c5a0 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 msgr2=0x7f50d403e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 0x7f50d403e990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d8003a10 tx=0x7f50d8009290 comp rx=0 tx=0).stop 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 msgr2=0x7f50ec19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50e0011a70 tx=0x7f50e0011b50 comp rx=0 tx=0).stop 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 shutdown_connections 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f50d403c5a0 0x7f50d403e990 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50ec07ce60 0x7f50ec19d6a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.223 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 --2- 172.21.15.72:0/386297550 >> [v2:172.21.15.72:6800/1095172224,v1:172.21.15.72:6801/1095172224] conn(0x7f50d4038cb0 0x7f50d403b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:58.224 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 >> 172.21.15.72:0/386297550 conn(0x7f50ec102cd0 msgr2=0x7f50ec1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:58.224 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 shutdown_connections 2024-03-11T19:10:58.224 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:57.590+0000 7f50f3800700 1 -- 172.21.15.72:0/386297550 wait complete. 2024-03-11T19:10:58.224 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 5 is available 2024-03-11T19:10:58.224 INFO:teuthology.orchestra.run.smithi072.stdout:Setting orchestrator backend to cephadm... 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: [11/Mar/2024:19:10:57] ENGINE Bus STARTING 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: [11/Mar/2024:19:10:57] ENGINE Serving on https://172.21.15.72:7150 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: [11/Mar/2024:19:10:57] ENGINE Serving on http://172.21.15.72:8765 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: [11/Mar/2024:19:10:57] ENGINE Bus STARTED 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:10:58.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:58 smithi072 ceph-mon[31695]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:10:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:59 smithi072 ceph-mon[31695]: mgrmap e8: x(active, since 2s) 2024-03-11T19:10:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:59 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:10:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:10:59 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.829+0000 7f6744000700 1 Processor -- start 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.829+0000 7f6744000700 1 -- start start 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.830+0000 7f6744000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.830+0000 7f6744000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c108080 con 0x7f673c107690 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.830+0000 7f6741600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.830+0000 7f6741600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40118/0 (socket says 172.21.15.72:40118) 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.830+0000 7f6741600700 1 -- 172.21.15.72:0/1675408475 learned_addr learned my addr 172.21.15.72:0/1675408475 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.831+0000 7f6741600700 1 -- 172.21.15.72:0/1675408475 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c108840 con 0x7f673c107690 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.831+0000 7f6741600700 1 --2- 172.21.15.72:0/1675408475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67300098a0 tx=0x7f6730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5caae0ea3aa72f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.831+0000 7f673be00700 1 -- 172.21.15.72:0/1675408475 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6730004160 con 0x7f673c107690 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.831+0000 7f673be00700 1 -- 172.21.15.72:0/1675408475 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67300042c0 con 0x7f673c107690 2024-03-11T19:10:59.725 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f673be00700 1 -- 172.21.15.72:0/1675408475 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6730011ac0 con 0x7f673c107690 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 -- 172.21.15.72:0/1675408475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 msgr2=0x7f673c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 --2- 172.21.15.72:0/1675408475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67300098a0 tx=0x7f6730009bb0 comp rx=0 tx=0).stop 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 -- 172.21.15.72:0/1675408475 shutdown_connections 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 --2- 172.21.15.72:0/1675408475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c107ab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 -- 172.21.15.72:0/1675408475 >> 172.21.15.72:0/1675408475 conn(0x7f673c102cd0 msgr2=0x7f673c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 -- 172.21.15.72:0/1675408475 shutdown_connections 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.832+0000 7f6744000700 1 -- 172.21.15.72:0/1675408475 wait complete. 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.833+0000 7f6744000700 1 Processor -- start 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6744000700 1 -- start start 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6744000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6744000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c07d0c0 con 0x7f673c107690 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6741600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6741600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40120/0 (socket says 172.21.15.72:40120) 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.834+0000 7f6741600700 1 -- 172.21.15.72:0/2157385812 learned_addr learned my addr 172.21.15.72:0/2157385812 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:10:59.726 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f6741600700 1 -- 172.21.15.72:0/2157385812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6730009550 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f6741600700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6730003f70 tx=0x7f6730011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67300039f0 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6730003b50 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c07b2b0 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f673001a690 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.835+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c07b750 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.836+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f673001a960 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.836+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c10ba70 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.837+0000 7f673a000700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 0x7f672403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.837+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f673004d0f0 con 0x7f673c107690 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.837+0000 7f6740c00700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 0x7f672403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.838+0000 7f6740c00700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 0x7f672403b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6728006fd0 tx=0x7f6728006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:10:59.727 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:58.843+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6730080d20 con 0x7f673c107690 2024-03-11T19:10:59.728 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.078+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f673c07be40 con 0x7f6724038cb0 2024-03-11T19:10:59.728 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.091+0000 7f673a000700 1 -- 172.21.15.72:0/2157385812 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f673c07be40 con 0x7f6724038cb0 2024-03-11T19:10:59.728 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 msgr2=0x7f672403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:59.728 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 0x7f672403b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6728006fd0 tx=0x7f6728006e40 comp rx=0 tx=0).stop 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 msgr2=0x7f673c07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6730003f70 tx=0x7f6730011b40 comp rx=0 tx=0).stop 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 shutdown_connections 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f6724038cb0 0x7f672403b170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 --2- 172.21.15.72:0/2157385812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f673c107690 0x7f673c07cb80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 >> 172.21.15.72:0/2157385812 conn(0x7f673c102cd0 msgr2=0x7f673c1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 shutdown_connections 2024-03-11T19:10:59.729 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:10:59.095+0000 7f6744000700 1 -- 172.21.15.72:0/2157385812 wait complete. 2024-03-11T19:11:00.887 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:00 smithi072 ceph-mon[31695]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:01.270 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.308+0000 7f033e000700 1 Processor -- start 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.309+0000 7f033e000700 1 -- start start 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.309+0000 7f033e000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.309+0000 7f033e000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033810a0e0 con 0x7f03381096f0 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.310+0000 7f033cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.310+0000 7f033cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40124/0 (socket says 172.21.15.72:40124) 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.310+0000 7f033cc00700 1 -- 172.21.15.72:0/3889511170 learned_addr learned my addr 172.21.15.72:0/3889511170 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.310+0000 7f033cc00700 1 -- 172.21.15.72:0/3889511170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033810a8a0 con 0x7f03381096f0 2024-03-11T19:11:01.271 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.311+0000 7f033cc00700 1 --2- 172.21.15.72:0/3889511170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098a0 tx=0x7f032c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f2f99e8c25c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.311+0000 7f0337400700 1 -- 172.21.15.72:0/3889511170 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c004160 con 0x7f03381096f0 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.311+0000 7f0337400700 1 -- 172.21.15.72:0/3889511170 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f032c0042c0 con 0x7f03381096f0 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.311+0000 7f0337400700 1 -- 172.21.15.72:0/3889511170 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c011ac0 con 0x7f03381096f0 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 -- 172.21.15.72:0/3889511170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 msgr2=0x7f0338109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 --2- 172.21.15.72:0/3889511170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098a0 tx=0x7f032c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 -- 172.21.15.72:0/3889511170 shutdown_connections 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 --2- 172.21.15.72:0/3889511170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f0338109b10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 -- 172.21.15.72:0/3889511170 >> 172.21.15.72:0/3889511170 conn(0x7f033807cc20 msgr2=0x7f03381074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 -- 172.21.15.72:0/3889511170 shutdown_connections 2024-03-11T19:11:01.272 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.312+0000 7f033e000700 1 -- 172.21.15.72:0/3889511170 wait complete. 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.313+0000 7f033e000700 1 Processor -- start 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.313+0000 7f033e000700 1 -- start start 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033e000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033e000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033819d9e0 con 0x7f03381096f0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40138/0 (socket says 172.21.15.72:40138) 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033cc00700 1 -- 172.21.15.72:0/1025148806 learned_addr learned my addr 172.21.15.72:0/1025148806 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.314+0000 7f033cc00700 1 -- 172.21.15.72:0/1025148806 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009550 con 0x7f03381096f0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f033cc00700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f032c009fb0 tx=0x7f032c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c0038b0 con 0x7f03381096f0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f032c003a10 con 0x7f03381096f0 2024-03-11T19:11:01.273 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033819dbe0 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c0195e0 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.315+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033819e080 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.316+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033810dad0 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.316+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f032c01a430 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.317+0000 7f0335600700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 0x7f032003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.317+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f032c04ce80 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.317+0000 7f0337e00700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 0x7f032003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.318+0000 7f0337e00700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 0x7f032003b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.322+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f032c003b80 con 0x7f03381096f0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.541+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f033810dad0 con 0x7f0320038cb0 2024-03-11T19:11:01.274 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.543+0000 7f0335600700 1 -- 172.21.15.72:0/1025148806 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f033810dad0 con 0x7f0320038cb0 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.545+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 msgr2=0x7f032003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.545+0000 7f033e000700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 0x7f032003b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0324006fd0 tx=0x7f0324006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.545+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 msgr2=0x7f033819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.545+0000 7f033e000700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f032c009fb0 tx=0x7f032c011b20 comp rx=0 tx=0).stop 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 shutdown_connections 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0320038cb0 0x7f032003b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 --2- 172.21.15.72:0/1025148806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03381096f0 0x7f033819d4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 >> 172.21.15.72:0/1025148806 conn(0x7f033807cc20 msgr2=0x7f0338106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 shutdown_connections 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:00.546+0000 7f033e000700 1 -- 172.21.15.72:0/1025148806 wait complete. 2024-03-11T19:11:01.275 INFO:teuthology.orchestra.run.smithi072.stdout:Generating ssh key... 2024-03-11T19:11:01.816 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:01 smithi072 ceph-mon[31695]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:02.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: Generating public/private rsa key pair. 2024-03-11T19:11:02.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: Your identification has been saved in /tmp/tmpi610b48l/key. 2024-03-11T19:11:02.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: Your public key has been saved in /tmp/tmpi610b48l/key.pub. 2024-03-11T19:11:02.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: The key fingerprint is: 2024-03-11T19:11:02.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: SHA256:VXSTy8Cmn7YuEIvzQjqVyTVr5v9GMnYLRdrpLJqKMf8 ceph-d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: The key's randomart image is: 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: +---[RSA 3072]----+ 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | oo o. | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | .=... | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | .* + . | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | +.o + o | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | . =S= = . | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | O * * O | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | o+ * = O o | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | o=. = . + | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: | ..o+E..=o | 2024-03-11T19:11:02.431 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: +----[SHA256]-----+ 2024-03-11T19:11:02.883 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.812+0000 7fee69600700 1 Processor -- start 2024-03-11T19:11:02.883 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.812+0000 7fee69600700 1 -- start start 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.813+0000 7fee69600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.813+0000 7fee69600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641080b0 con 0x7fee641076c0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.813+0000 7fee62a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.813+0000 7fee62a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40146/0 (socket says 172.21.15.72:40146) 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.813+0000 7fee62a00700 1 -- 172.21.15.72:0/3232166228 learned_addr learned my addr 172.21.15.72:0/3232166228 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.814+0000 7fee62a00700 1 -- 172.21.15.72:0/3232166228 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64108870 con 0x7fee641076c0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.814+0000 7fee62a00700 1 --2- 172.21.15.72:0/3232166228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee540098a0 tx=0x7fee54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2b55b711646e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.814+0000 7fee61600700 1 -- 172.21.15.72:0/3232166228 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee54004160 con 0x7fee641076c0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.814+0000 7fee61600700 1 -- 172.21.15.72:0/3232166228 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee540042c0 con 0x7fee641076c0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.814+0000 7fee61600700 1 -- 172.21.15.72:0/3232166228 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee54011ac0 con 0x7fee641076c0 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 -- 172.21.15.72:0/3232166228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 msgr2=0x7fee64107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:02.884 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 --2- 172.21.15.72:0/3232166228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee540098a0 tx=0x7fee54009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 -- 172.21.15.72:0/3232166228 shutdown_connections 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 --2- 172.21.15.72:0/3232166228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee64107ae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 -- 172.21.15.72:0/3232166228 >> 172.21.15.72:0/3232166228 conn(0x7fee64102d00 msgr2=0x7fee64105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 -- 172.21.15.72:0/3232166228 shutdown_connections 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.815+0000 7fee69600700 1 -- 172.21.15.72:0/3232166228 wait complete. 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.816+0000 7fee69600700 1 Processor -- start 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.816+0000 7fee69600700 1 -- start start 2024-03-11T19:11:02.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee69600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee69600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419b9f0 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee62a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee62a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40158/0 (socket says 172.21.15.72:40158) 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee62a00700 1 -- 172.21.15.72:0/3783362387 learned_addr learned my addr 172.21.15.72:0/3783362387 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.817+0000 7fee62a00700 1 -- 172.21.15.72:0/3783362387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009550 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee62a00700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee54011780 tx=0x7fee54011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee54003af0 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee54003c50 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6419bbf0 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee540197e0 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.818+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6419c090 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.819+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fee5401a430 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.819+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6410baa0 con 0x7fee641076c0 2024-03-11T19:11:02.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.820+0000 7fee5b400700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 0x7fee4803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.820+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fee5404cf60 con 0x7fee641076c0 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.820+0000 7fee62000700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 0x7fee4803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.820+0000 7fee62000700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 0x7fee4803b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:01.825+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee5402da40 con 0x7fee641076c0 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.044+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fee64103f60 con 0x7fee48038d00 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.196+0000 7fee5b400700 1 -- 172.21.15.72:0/3783362387 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fee64103f60 con 0x7fee48038d00 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 msgr2=0x7fee4803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 0x7fee4803b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 msgr2=0x7fee6419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee54011780 tx=0x7fee54011b40 comp rx=0 tx=0).stop 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 shutdown_connections 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fee48038d00 0x7fee4803b1c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 --2- 172.21.15.72:0/3783362387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee641076c0 0x7fee6419b4b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:02.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.199+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 >> 172.21.15.72:0/3783362387 conn(0x7fee64102d00 msgr2=0x7fee64103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:02.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.200+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 shutdown_connections 2024-03-11T19:11:02.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:02.200+0000 7fee69600700 1 -- 172.21.15.72:0/3783362387 wait complete. 2024-03-11T19:11:03.501 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:03 smithi072 ceph-mon[31695]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:03.501 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:03 smithi072 ceph-mon[31695]: Generating ssh key... 2024-03-11T19:11:03.501 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:03 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:03.501 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:03 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.496+0000 7f5bf7000700 1 Processor -- start 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.497+0000 7f5bf7000700 1 -- start start 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.497+0000 7f5bf7000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.497+0000 7f5bf7000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0106d60 con 0x7f5bf0106370 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.497+0000 7f5bf5c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:04.333 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.497+0000 7f5bf5c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55216/0 (socket says 172.21.15.72:55216) 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.498+0000 7f5bf5c00700 1 -- 172.21.15.72:0/2415416059 learned_addr learned my addr 172.21.15.72:0/2415416059 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.498+0000 7f5bf5c00700 1 -- 172.21.15.72:0/2415416059 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0107570 con 0x7f5bf0106370 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.498+0000 7f5bf5c00700 1 --2- 172.21.15.72:0/2415416059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098a0 tx=0x7f5be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21363db3db4155c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.498+0000 7f5befe00700 1 -- 172.21.15.72:0/2415416059 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be4004160 con 0x7f5bf0106370 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5befe00700 1 -- 172.21.15.72:0/2415416059 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5be40042c0 con 0x7f5bf0106370 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5befe00700 1 -- 172.21.15.72:0/2415416059 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be4011ac0 con 0x7f5bf0106370 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 -- 172.21.15.72:0/2415416059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 msgr2=0x7f5bf0106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2415416059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098a0 tx=0x7f5be4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 -- 172.21.15.72:0/2415416059 shutdown_connections 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2415416059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0106790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 -- 172.21.15.72:0/2415416059 >> 172.21.15.72:0/2415416059 conn(0x7f5bf0101970 msgr2=0x7f5bf0103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:04.334 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.499+0000 7f5bf7000700 1 -- 172.21.15.72:0/2415416059 shutdown_connections 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.500+0000 7f5bf7000700 1 -- 172.21.15.72:0/2415416059 wait complete. 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf7000700 1 Processor -- start 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf7000700 1 -- start start 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf7000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf7000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf01919d0 con 0x7f5bf0106370 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf5c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf5c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55228/0 (socket says 172.21.15.72:55228) 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.501+0000 7f5bf5c00700 1 -- 172.21.15.72:0/2114287307 learned_addr learned my addr 172.21.15.72:0/2114287307 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bf5c00700 1 -- 172.21.15.72:0/2114287307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009550 con 0x7f5bf0106370 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bf5c00700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000c00 tx=0x7f5be4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be4003aa0 con 0x7f5bf0106370 2024-03-11T19:11:04.335 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5be4003c00 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf018fc10 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be40248a0 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.502+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf01900b0 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.504+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf0068d60 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.504+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f5be401ac60 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.504+0000 7f5bee000700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 0x7f5bd803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.505+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5be404cdd0 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.505+0000 7f5bf5200700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 0x7f5bd803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.505+0000 7f5bf5200700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 0x7f5bd803b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc006fd0 tx=0x7f5bdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.509+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5be401a430 con 0x7f5bf0106370 2024-03-11T19:11:04.336 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.728+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bf0194fe0 con 0x7f5bd8038d00 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.730+0000 7f5bee000700 1 -- 172.21.15.72:0/2114287307 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f5bf0194fe0 con 0x7f5bd8038d00 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.733+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 msgr2=0x7f5bd803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.733+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 0x7f5bd803b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc006fd0 tx=0x7f5bdc006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.733+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 msgr2=0x7f5bf0191490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.733+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000c00 tx=0x7f5be4011b40 comp rx=0 tx=0).stop 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 shutdown_connections 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f5bd8038d00 0x7f5bd803b1c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 --2- 172.21.15.72:0/2114287307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0106370 0x7f5bf0191490 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 >> 172.21.15.72:0/2114287307 conn(0x7f5bf0101970 msgr2=0x7f5bf0102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 shutdown_connections 2024-03-11T19:11:04.337 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:03.734+0000 7f5bf7000700 1 -- 172.21.15.72:0/2114287307 wait complete. 2024-03-11T19:11:04.338 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:11:04.338 INFO:teuthology.orchestra.run.smithi072.stdout:Adding key to root@localhost authorized_keys... 2024-03-11T19:11:04.338 INFO:teuthology.orchestra.run.smithi072.stdout:Adding host smithi072... 2024-03-11T19:11:04.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:04 smithi072 ceph-mon[31695]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:06.300 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:06 smithi072 ceph-mon[31695]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:07.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:07 smithi072 ceph-mon[31695]: Deploying cephadm binary to smithi072 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Added host 'smithi072' with addr '172.21.15.72' 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.939+0000 7f89ebc00700 1 Processor -- start 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.939+0000 7f89ebc00700 1 -- start start 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.939+0000 7f89ebc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.939+0000 7f89ebc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4108080 con 0x7f89e4107690 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.940+0000 7f89e9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.940+0000 7f89e9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55238/0 (socket says 172.21.15.72:55238) 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.940+0000 7f89e9200700 1 -- 172.21.15.72:0/2659178276 learned_addr learned my addr 172.21.15.72:0/2659178276 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.940+0000 7f89e9200700 1 -- 172.21.15.72:0/2659178276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4108840 con 0x7f89e4107690 2024-03-11T19:11:12.103 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89e9200700 1 --2- 172.21.15.72:0/2659178276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098a0 tx=0x7f89d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e16e2adc89eed97 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89e3400700 1 -- 172.21.15.72:0/2659178276 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89d8004160 con 0x7f89e4107690 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89e3400700 1 -- 172.21.15.72:0/2659178276 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89d80042c0 con 0x7f89e4107690 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89e3400700 1 -- 172.21.15.72:0/2659178276 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89d8011ac0 con 0x7f89e4107690 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89ebc00700 1 -- 172.21.15.72:0/2659178276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 msgr2=0x7f89e4107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.941+0000 7f89ebc00700 1 --2- 172.21.15.72:0/2659178276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098a0 tx=0x7f89d8009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.942+0000 7f89ebc00700 1 -- 172.21.15.72:0/2659178276 shutdown_connections 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.942+0000 7f89ebc00700 1 --2- 172.21.15.72:0/2659178276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e4107ab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.942+0000 7f89ebc00700 1 -- 172.21.15.72:0/2659178276 >> 172.21.15.72:0/2659178276 conn(0x7f89e4102cd0 msgr2=0x7f89e41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.942+0000 7f89ebc00700 1 -- 172.21.15.72:0/2659178276 shutdown_connections 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.942+0000 7f89ebc00700 1 -- 172.21.15.72:0/2659178276 wait complete. 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89ebc00700 1 Processor -- start 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89ebc00700 1 -- start start 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89ebc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89ebc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e419db60 con 0x7f89e4107690 2024-03-11T19:11:12.104 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89e9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89e9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55240/0 (socket says 172.21.15.72:55240) 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.943+0000 7f89e9200700 1 -- 172.21.15.72:0/1598150954 learned_addr learned my addr 172.21.15.72:0/1598150954 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89e9200700 1 -- 172.21.15.72:0/1598150954 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009550 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89e9200700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89d8003aa0 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89d8003c00 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e419dd60 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.944+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89d80248a0 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.945+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e419e200 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.946+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f89d801ac60 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.946+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e410ba70 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.946+0000 7f89e1600700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 0x7f89cc03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.946+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f89d804ce40 con 0x7f89e4107690 2024-03-11T19:11:12.105 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.946+0000 7f89e3e00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 0x7f89cc03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.947+0000 7f89e3e00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 0x7f89cc03b1c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:04.952+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89d8018530 con 0x7f89e4107690 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:05.175+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}) v1 -- 0x7f89e4103e40 con 0x7f89cc038d00 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.440+0000 7f89e1600700 1 -- 172.21.15.72:0/1598150954 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f89e4103e40 con 0x7f89cc038d00 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 msgr2=0x7f89cc03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 0x7f89cc03b1c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 msgr2=0x7f89e419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89d8000c00 tx=0x7f89d8011b40 comp rx=0 tx=0).stop 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 shutdown_connections 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f89cc038d00 0x7f89cc03b1c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 --2- 172.21.15.72:0/1598150954 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e4107690 0x7f89e419d620 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 >> 172.21.15.72:0/1598150954 conn(0x7f89e4102cd0 msgr2=0x7f89e4103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.445+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 shutdown_connections 2024-03-11T19:11:12.106 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:11.446+0000 7f89ebc00700 1 -- 172.21.15.72:0/1598150954 wait complete. 2024-03-11T19:11:12.107 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mon service... 2024-03-11T19:11:12.659 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:12 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:12.659 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:12 smithi072 ceph-mon[31695]: Added host smithi072 2024-03-11T19:11:12.659 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:12 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:13.853 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-11T19:11:13.853 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.688+0000 7ff78b200700 1 Processor -- start 2024-03-11T19:11:13.853 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.688+0000 7ff78b200700 1 -- start start 2024-03-11T19:11:13.853 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.688+0000 7ff78b200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:13.854 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.688+0000 7ff78b200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c107ee0 con 0x7ff78c1074f0 2024-03-11T19:11:13.854 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.688+0000 7ff789e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:13.854 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.689+0000 7ff789e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36488/0 (socket says 172.21.15.72:36488) 2024-03-11T19:11:13.854 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.689+0000 7ff789e00700 1 -- 172.21.15.72:0/1356893626 learned_addr learned my addr 172.21.15.72:0/1356893626 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:13.854 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.689+0000 7ff789e00700 1 -- 172.21.15.72:0/1356893626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c1086a0 con 0x7ff78c1074f0 2024-03-11T19:11:13.855 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.689+0000 7ff789e00700 1 --2- 172.21.15.72:0/1356893626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098a0 tx=0x7ff77c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17e492cedf6f7db server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:13.855 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.689+0000 7ff788a00700 1 -- 172.21.15.72:0/1356893626 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff77c004160 con 0x7ff78c1074f0 2024-03-11T19:11:13.855 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff788a00700 1 -- 172.21.15.72:0/1356893626 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff77c0042c0 con 0x7ff78c1074f0 2024-03-11T19:11:13.855 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff788a00700 1 -- 172.21.15.72:0/1356893626 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff77c011ac0 con 0x7ff78c1074f0 2024-03-11T19:11:13.855 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 -- 172.21.15.72:0/1356893626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 msgr2=0x7ff78c107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 --2- 172.21.15.72:0/1356893626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098a0 tx=0x7ff77c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 -- 172.21.15.72:0/1356893626 shutdown_connections 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 --2- 172.21.15.72:0/1356893626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c107910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 -- 172.21.15.72:0/1356893626 >> 172.21.15.72:0/1356893626 conn(0x7ff78c102b10 msgr2=0x7ff78c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.690+0000 7ff78b200700 1 -- 172.21.15.72:0/1356893626 shutdown_connections 2024-03-11T19:11:13.856 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.691+0000 7ff78b200700 1 -- 172.21.15.72:0/1356893626 wait complete. 2024-03-11T19:11:13.857 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.691+0000 7ff78b200700 1 Processor -- start 2024-03-11T19:11:13.857 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff78b200700 1 -- start start 2024-03-11T19:11:13.857 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff78b200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:13.857 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff78b200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c19b7e0 con 0x7ff78c1074f0 2024-03-11T19:11:13.857 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff789e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:13.858 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff789e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36492/0 (socket says 172.21.15.72:36492) 2024-03-11T19:11:13.858 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.692+0000 7ff789e00700 1 -- 172.21.15.72:0/1889017680 learned_addr learned my addr 172.21.15.72:0/1889017680 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:13.858 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff789e00700 1 -- 172.21.15.72:0/1889017680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c009550 con 0x7ff78c1074f0 2024-03-11T19:11:13.861 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff789e00700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff77c011780 tx=0x7ff77c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:13.862 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff77c003af0 con 0x7ff78c1074f0 2024-03-11T19:11:13.862 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff77c003c50 con 0x7ff78c1074f0 2024-03-11T19:11:13.862 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c19b9e0 con 0x7ff78c1074f0 2024-03-11T19:11:13.862 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff77c0197e0 con 0x7ff78c1074f0 2024-03-11T19:11:13.862 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.693+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c19be80 con 0x7ff78c1074f0 2024-03-11T19:11:13.863 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.694+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c10b8d0 con 0x7ff78c1074f0 2024-03-11T19:11:13.863 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.694+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7ff77c01aba0 con 0x7ff78c1074f0 2024-03-11T19:11:13.863 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.695+0000 7ff782a00700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 0x7ff77003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:13.863 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.695+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff77c04d060 con 0x7ff78c1074f0 2024-03-11T19:11:13.863 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.695+0000 7ff789400700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 0x7ff77003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.696+0000 7ff789400700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 0x7ff77003b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.700+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff77c01a430 con 0x7ff78c1074f0 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.938+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff78c103d10 con 0x7ff770038d00 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.946+0000 7ff782a00700 1 -- 172.21.15.72:0/1889017680 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff78c103d10 con 0x7ff770038d00 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 msgr2=0x7ff77003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:13.864 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 0x7ff77003b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:13.865 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 msgr2=0x7ff78c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:13.865 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff77c011780 tx=0x7ff77c011b40 comp rx=0 tx=0).stop 2024-03-11T19:11:13.865 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 shutdown_connections 2024-03-11T19:11:13.865 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7ff770038d00 0x7ff77003b1c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:13.865 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 --2- 172.21.15.72:0/1889017680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff78c1074f0 0x7ff78c19b2a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:13.866 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 >> 172.21.15.72:0/1889017680 conn(0x7ff78c102b10 msgr2=0x7ff78c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:13.866 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 shutdown_connections 2024-03-11T19:11:13.866 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:12.949+0000 7ff78b200700 1 -- 172.21.15.72:0/1889017680 wait complete. 2024-03-11T19:11:13.866 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mgr service... 2024-03-11T19:11:14.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:13 smithi072 ceph-mon[31695]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:14.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:13 smithi072 ceph-mon[31695]: Saving service mon spec with placement count:5 2024-03-11T19:11:14.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:13 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:15 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:15 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:15 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:16.040 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.121+0000 7fe0f0600700 1 Processor -- start 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.121+0000 7fe0f0600700 1 -- start start 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.121+0000 7fe0f0600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.121+0000 7fe0f0600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e807b8e0 con 0x7fe0e807ce60 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.122+0000 7fe0edc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.122+0000 7fe0edc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36498/0 (socket says 172.21.15.72:36498) 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.122+0000 7fe0edc00700 1 -- 172.21.15.72:0/3388115279 learned_addr learned my addr 172.21.15.72:0/3388115279 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.122+0000 7fe0edc00700 1 -- 172.21.15.72:0/3388115279 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e807ba20 con 0x7fe0e807ce60 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.123+0000 7fe0edc00700 1 --2- 172.21.15.72:0/3388115279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0098a0 tx=0x7fe0dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e66ef2a16245309 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.123+0000 7fe0e7e00700 1 -- 172.21.15.72:0/3388115279 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0dc004160 con 0x7fe0e807ce60 2024-03-11T19:11:16.041 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.123+0000 7fe0e7e00700 1 -- 172.21.15.72:0/3388115279 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc0042c0 con 0x7fe0e807ce60 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.123+0000 7fe0e7e00700 1 -- 172.21.15.72:0/3388115279 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0dc011a80 con 0x7fe0e807ce60 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 -- 172.21.15.72:0/3388115279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 msgr2=0x7fe0e807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 --2- 172.21.15.72:0/3388115279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0098a0 tx=0x7fe0dc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 -- 172.21.15.72:0/3388115279 shutdown_connections 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 --2- 172.21.15.72:0/3388115279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e807b310 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 -- 172.21.15.72:0/3388115279 >> 172.21.15.72:0/3388115279 conn(0x7fe0e8102d30 msgr2=0x7fe0e8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 -- 172.21.15.72:0/3388115279 shutdown_connections 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.124+0000 7fe0f0600700 1 -- 172.21.15.72:0/3388115279 wait complete. 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.125+0000 7fe0f0600700 1 Processor -- start 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.125+0000 7fe0f0600700 1 -- start start 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.125+0000 7fe0f0600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.125+0000 7fe0f0600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e819db50 con 0x7fe0e807ce60 2024-03-11T19:11:16.042 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.126+0000 7fe0edc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.126+0000 7fe0edc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36506/0 (socket says 172.21.15.72:36506) 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.126+0000 7fe0edc00700 1 -- 172.21.15.72:0/2099954075 learned_addr learned my addr 172.21.15.72:0/2099954075 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.126+0000 7fe0edc00700 1 -- 172.21.15.72:0/2099954075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc009550 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.126+0000 7fe0edc00700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003f20 tx=0x7fe0dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.127+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0dc0039f0 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.127+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc003b50 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.127+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e819dd50 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.127+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0dc01a6d0 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.128+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e819e1f0 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.128+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe0dc01a960 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.129+0000 7fe0e6000700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 0x7fe0d003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.129+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe0dc04e870 con 0x7fe0e807ce60 2024-03-11T19:11:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.129+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e810daf0 con 0x7fe0e807ce60 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.129+0000 7fe0ed200700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 0x7fe0d003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.130+0000 7fe0ed200700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 0x7fe0d003b170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006fd0 tx=0x7fe0d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.134+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0dc019ac0 con 0x7fe0e807ce60 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.367+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe0e8103f30 con 0x7fe0d0038cb0 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.374+0000 7fe0e6000700 1 -- 172.21.15.72:0/2099954075 <== mgr.14120 v2:172.21.15.72:6800/2769969675 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe0e8103f30 con 0x7fe0d0038cb0 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 msgr2=0x7fe0d003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 0x7fe0d003b170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006fd0 tx=0x7fe0d4006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 msgr2=0x7fe0e819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003f20 tx=0x7fe0dc011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 shutdown_connections 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe0d0038cb0 0x7fe0d003b170 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 --2- 172.21.15.72:0/2099954075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0e807ce60 0x7fe0e819d610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 >> 172.21.15.72:0/2099954075 conn(0x7fe0e8102d30 msgr2=0x7fe0e8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:16.044 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 shutdown_connections 2024-03-11T19:11:16.045 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:15.376+0000 7fe0f0600700 1 -- 172.21.15.72:0/2099954075 wait complete. 2024-03-11T19:11:16.839 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:16 smithi072 ceph-mon[31695]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:16.839 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:16 smithi072 ceph-mon[31695]: Saving service mgr spec with placement count:2 2024-03-11T19:11:17.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:17 smithi072 ceph-mon[31695]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:17.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:17 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1955252028' entity='client.admin' 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.582+0000 7f0bc5e00700 1 Processor -- start 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.583+0000 7f0bc5e00700 1 -- start start 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.583+0000 7f0bc5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.583+0000 7f0bc5e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc007ae80 con 0x7f0bc0105f30 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.583+0000 7f0bbea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.584+0000 7f0bbea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36508/0 (socket says 172.21.15.72:36508) 2024-03-11T19:11:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.584+0000 7f0bbea00700 1 -- 172.21.15.72:0/456861604 learned_addr learned my addr 172.21.15.72:0/456861604 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.584+0000 7f0bbea00700 1 -- 172.21.15.72:0/456861604 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc01088a0 con 0x7f0bc0105f30 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.584+0000 7f0bbea00700 1 --2- 172.21.15.72:0/456861604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098a0 tx=0x7f0bb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9cb191a02bd6d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.584+0000 7f0bbd600700 1 -- 172.21.15.72:0/456861604 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0004160 con 0x7f0bc0105f30 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bbd600700 1 -- 172.21.15.72:0/456861604 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bb00042c0 con 0x7f0bc0105f30 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bbd600700 1 -- 172.21.15.72:0/456861604 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0011a80 con 0x7f0bc0105f30 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 -- 172.21.15.72:0/456861604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 msgr2=0x7f0bc0108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 --2- 172.21.15.72:0/456861604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098a0 tx=0x7f0bb0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 -- 172.21.15.72:0/456861604 shutdown_connections 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 --2- 172.21.15.72:0/456861604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc0108360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 -- 172.21.15.72:0/456861604 >> 172.21.15.72:0/456861604 conn(0x7f0bc0101b90 msgr2=0x7f0bc0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 -- 172.21.15.72:0/456861604 shutdown_connections 2024-03-11T19:11:17.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.585+0000 7f0bc5e00700 1 -- 172.21.15.72:0/456861604 wait complete. 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.586+0000 7f0bc5e00700 1 Processor -- start 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bc5e00700 1 -- start start 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bc5e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bc5e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a1f70 con 0x7f0bc0105f30 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bbea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bbea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36518/0 (socket says 172.21.15.72:36518) 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.587+0000 7f0bbea00700 1 -- 172.21.15.72:0/1955252028 learned_addr learned my addr 172.21.15.72:0/1955252028 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bbea00700 1 -- 172.21.15.72:0/1955252028 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0009550 con 0x7f0bc0105f30 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bbea00700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009130 tx=0x7f0bb0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0003aa0 con 0x7f0bc0105f30 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0003c00 con 0x7f0bc0105f30 2024-03-11T19:11:17.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bc5e00700 1 -- 172.21.15.72:0/1955252028 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc01a2170 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bb00248a0 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.588+0000 7f0bc5e00700 1 -- 172.21.15.72:0/1955252028 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc01a2610 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.590+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f0bb001a430 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.590+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b98005330 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.590+0000 7f0bc4a00700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 0x7f0ba403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.590+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0bb004ce60 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.590+0000 7f0bbe000700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 0x7f0ba403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.591+0000 7f0bbe000700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 0x7f0ba403b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bac006fd0 tx=0x7f0bac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.596+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bb001a6e0 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.829+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0b980055e0 con 0x7f0bc0105f30 2024-03-11T19:11:17.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.835+0000 7f0bc4a00700 1 -- 172.21.15.72:0/1955252028 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0bb000b3e0 con 0x7f0bc0105f30 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 msgr2=0x7f0ba403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 0x7f0ba403b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bac006fd0 tx=0x7f0bac006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 msgr2=0x7f0bc01a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009130 tx=0x7f0bb0011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 shutdown_connections 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f0ba4038d00 0x7f0ba403b1c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 --2- 172.21.15.72:0/1955252028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0bc0105f30 0x7f0bc01a1a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.838+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 >> 172.21.15.72:0/1955252028 conn(0x7f0bc0101b90 msgr2=0x7f0bc0103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.839+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 shutdown_connections 2024-03-11T19:11:17.979 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:16.839+0000 7f0bb5600700 1 -- 172.21.15.72:0/1955252028 wait complete. 2024-03-11T19:11:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:19 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:19 smithi072 ceph-mon[31695]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:19 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/164816420' entity='client.admin' 2024-03-11T19:11:19.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.560+0000 7f71a7600700 1 Processor -- start 2024-03-11T19:11:19.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.560+0000 7f71a7600700 1 -- start start 2024-03-11T19:11:19.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.560+0000 7f71a7600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:19.524 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.560+0000 7f71a7600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0108060 con 0x7f71a0107670 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.561+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.561+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36524/0 (socket says 172.21.15.72:36524) 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.561+0000 7f71a4c00700 1 -- 172.21.15.72:0/3031319120 learned_addr learned my addr 172.21.15.72:0/3031319120 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.561+0000 7f71a4c00700 1 -- 172.21.15.72:0/3031319120 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0108820 con 0x7f71a0107670 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.562+0000 7f71a4c00700 1 --2- 172.21.15.72:0/3031319120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71940098a0 tx=0x7f7194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a11583b692b56af server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.562+0000 7f719f400700 1 -- 172.21.15.72:0/3031319120 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7194004160 con 0x7f71a0107670 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.562+0000 7f719f400700 1 -- 172.21.15.72:0/3031319120 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71940042c0 con 0x7f71a0107670 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.562+0000 7f719f400700 1 -- 172.21.15.72:0/3031319120 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7194011a80 con 0x7f71a0107670 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 -- 172.21.15.72:0/3031319120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 msgr2=0x7f71a0107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 --2- 172.21.15.72:0/3031319120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71940098a0 tx=0x7f7194009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 -- 172.21.15.72:0/3031319120 shutdown_connections 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 --2- 172.21.15.72:0/3031319120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a0107a90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 -- 172.21.15.72:0/3031319120 >> 172.21.15.72:0/3031319120 conn(0x7f71a0102cd0 msgr2=0x7f71a01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:19.525 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 -- 172.21.15.72:0/3031319120 shutdown_connections 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.563+0000 7f71a7600700 1 -- 172.21.15.72:0/3031319120 wait complete. 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.564+0000 7f71a7600700 1 Processor -- start 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.564+0000 7f71a7600700 1 -- start start 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.564+0000 7f71a7600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.564+0000 7f71a7600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a019db40 con 0x7f71a0107670 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.565+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.565+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36538/0 (socket says 172.21.15.72:36538) 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.565+0000 7f71a4c00700 1 -- 172.21.15.72:0/164816420 learned_addr learned my addr 172.21.15.72:0/164816420 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.565+0000 7f71a4c00700 1 -- 172.21.15.72:0/164816420 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009550 con 0x7f71a0107670 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.565+0000 7f71a4c00700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7194011740 tx=0x7f7194011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.566+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71940039e0 con 0x7f71a0107670 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.566+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7194003b40 con 0x7f71a0107670 2024-03-11T19:11:19.526 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.566+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7194024740 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.566+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a019dd40 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.566+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a019e1e0 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.567+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f719401a430 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.567+0000 7f719d600700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 0x7f718803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.567+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f719404cde0 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.567+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a010ba50 con 0x7f71a0107670 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.568+0000 7f719fe00700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 0x7f718803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:19.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.568+0000 7f719fe00700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 0x7f718803b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f718c006fd0 tx=0x7f718c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.573+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7194010380 con 0x7f71a0107670 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.821+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f71a010bc10 con 0x7f71a0107670 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.824+0000 7f719d600700 1 -- 172.21.15.72:0/164816420 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7194018be0 con 0x7f71a0107670 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 msgr2=0x7f718803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 0x7f718803b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f718c006fd0 tx=0x7f718c006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 msgr2=0x7f71a019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7194011740 tx=0x7f7194011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 shutdown_connections 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f7188038d00 0x7f718803b1c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 --2- 172.21.15.72:0/164816420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a0107670 0x7f71a019d600 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 >> 172.21.15.72:0/164816420 conn(0x7f71a0102cd0 msgr2=0x7f71a0103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 shutdown_connections 2024-03-11T19:11:19.528 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:18.828+0000 7f71a7600700 1 -- 172.21.15.72:0/164816420 wait complete. 2024-03-11T19:11:19.529 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling the dashboard module... 2024-03-11T19:11:21.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-mon[31695]: from='mgr.14120 172.21.15.72:0/4152239445' entity='mgr.x' 2024-03-11T19:11:21.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2830109524' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-11T19:11:21.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-mon[31695]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:21.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setuser ceph since I am not root 2024-03-11T19:11:21.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setgroup ceph since I am not root 2024-03-11T19:11:22.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:22 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2830109524' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-11T19:11:22.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:22 smithi072 ceph-mon[31695]: mgrmap e9: x(active, since 24s) 2024-03-11T19:11:22.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:21.841+0000 7f1f8cbd9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:11:22.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:21.968+0000 7f1f8cbd9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.055+0000 7fe4e2400700 1 Processor -- start 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.056+0000 7fe4e2400700 1 -- start start 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.056+0000 7fe4e2400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.056+0000 7fe4e2400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc07aec0 con 0x7fe4dc108130 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.057+0000 7fe4db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.057+0000 7fe4db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36546/0 (socket says 172.21.15.72:36546) 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.057+0000 7fe4db400700 1 -- 172.21.15.72:0/3281250878 learned_addr learned my addr 172.21.15.72:0/3281250878 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.057+0000 7fe4db400700 1 -- 172.21.15.72:0/3281250878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc10aa60 con 0x7fe4dc108130 2024-03-11T19:11:22.576 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.058+0000 7fe4db400700 1 --2- 172.21.15.72:0/3281250878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098a0 tx=0x7fe4cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8173feadc6dcb6fb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:22.577 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.058+0000 7fe4da000700 1 -- 172.21.15.72:0/3281250878 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc004160 con 0x7fe4dc108130 2024-03-11T19:11:22.577 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.058+0000 7fe4da000700 1 -- 172.21.15.72:0/3281250878 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc0042c0 con 0x7fe4dc108130 2024-03-11T19:11:22.577 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.058+0000 7fe4da000700 1 -- 172.21.15.72:0/3281250878 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc011a80 con 0x7fe4dc108130 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 -- 172.21.15.72:0/3281250878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 msgr2=0x7fe4dc10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 --2- 172.21.15.72:0/3281250878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098a0 tx=0x7fe4cc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 -- 172.21.15.72:0/3281250878 shutdown_connections 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 --2- 172.21.15.72:0/3281250878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc10a520 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 -- 172.21.15.72:0/3281250878 >> 172.21.15.72:0/3281250878 conn(0x7fe4dc101ba0 msgr2=0x7fe4dc103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.059+0000 7fe4e2400700 1 -- 172.21.15.72:0/3281250878 shutdown_connections 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.060+0000 7fe4e2400700 1 -- 172.21.15.72:0/3281250878 wait complete. 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4e2400700 1 Processor -- start 2024-03-11T19:11:22.582 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4e2400700 1 -- start start 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4e2400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4e2400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc19b980 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36556/0 (socket says 172.21.15.72:36556) 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.061+0000 7fe4db400700 1 -- 172.21.15.72:0/2830109524 learned_addr learned my addr 172.21.15.72:0/2830109524 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4db400700 1 -- 172.21.15.72:0/2830109524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009550 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4db400700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009130 tx=0x7fe4cc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc003aa0 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc003c00 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc0248a0 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.062+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc19bb80 con 0x7fe4dc108130 2024-03-11T19:11:22.583 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.063+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc19c020 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.064+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe4cc01ac10 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.064+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc10b900 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.064+0000 7fe4d3e00700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 0x7fe4c003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.064+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe4cc04cf00 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.065+0000 7fe4daa00700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 0x7fe4c003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.065+0000 7fe4daa00700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 0x7fe4c003b1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006fd0 tx=0x7fe4c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.069+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4cc024a60 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:20.359+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe4dc10bb10 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.050+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fe4cc02c350 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.052+0000 7fe4d3e00700 1 -- 172.21.15.72:0/2830109524 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fe4cc01a430 con 0x7fe4dc108130 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 msgr2=0x7fe4c003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 0x7fe4c003b1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006fd0 tx=0x7fe4c8006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:22.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 msgr2=0x7fe4dc19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009130 tx=0x7fe4cc011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 shutdown_connections 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7fe4c0038d00 0x7fe4c003b1c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 --2- 172.21.15.72:0/2830109524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe4dc108130 0x7fe4dc19b440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.055+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 >> 172.21.15.72:0/2830109524 conn(0x7fe4dc101ba0 msgr2=0x7fe4dc102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.056+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 shutdown_connections 2024-03-11T19:11:22.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:21.056+0000 7fe4e2400700 1 -- 172.21.15.72:0/2830109524 wait complete. 2024-03-11T19:11:23.030 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:22.764+0000 7f1f8cbd9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:11:23.030 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:22.889+0000 7f1f8cbd9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:11:23.315 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.067+0000 7f1f8cbd9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:11:23.316 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.199+0000 7f1f8cbd9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:11:23.316 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.311+0000 7f1f8cbd9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:11:23.654 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3140031666' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:11:23.906 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.649+0000 7f1f8cbd9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:11:23.907 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.790+0000 7f1f8cbd9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:11:23.907 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:23.902+0000 7f1f8cbd9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:11:24.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.180+0000 7f2c48000700 1 Processor -- start 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.180+0000 7f2c48000700 1 -- start start 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.181+0000 7f2c48000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.181+0000 7f2c48000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c401080a0 con 0x7f2c401076b0 2024-03-11T19:11:24.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.181+0000 7f2c45600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.181+0000 7f2c45600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49284/0 (socket says 172.21.15.72:49284) 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.181+0000 7f2c45600700 1 -- 172.21.15.72:0/2697381159 learned_addr learned my addr 172.21.15.72:0/2697381159 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.182+0000 7f2c45600700 1 -- 172.21.15.72:0/2697381159 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40108860 con 0x7f2c401076b0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.182+0000 7f2c45600700 1 --2- 172.21.15.72:0/2697381159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf0749de92ff59a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.182+0000 7f2c3fe00700 1 -- 172.21.15.72:0/2697381159 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c34004160 con 0x7f2c401076b0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.182+0000 7f2c3fe00700 1 -- 172.21.15.72:0/2697381159 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c340042c0 con 0x7f2c401076b0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.182+0000 7f2c3fe00700 1 -- 172.21.15.72:0/2697381159 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c34011a80 con 0x7f2c401076b0 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 -- 172.21.15.72:0/2697381159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 msgr2=0x7f2c40107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 --2- 172.21.15.72:0/2697381159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 -- 172.21.15.72:0/2697381159 shutdown_connections 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 --2- 172.21.15.72:0/2697381159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c40107ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:24.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 -- 172.21.15.72:0/2697381159 >> 172.21.15.72:0/2697381159 conn(0x7f2c40102c90 msgr2=0x7f2c401050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 -- 172.21.15.72:0/2697381159 shutdown_connections 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.183+0000 7f2c48000700 1 -- 172.21.15.72:0/2697381159 wait complete. 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.184+0000 7f2c48000700 1 Processor -- start 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.184+0000 7f2c48000700 1 -- start start 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c48000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c48000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4019b9e0 con 0x7f2c401076b0 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c45600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c45600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49292/0 (socket says 172.21.15.72:49292) 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c45600700 1 -- 172.21.15.72:0/3140031666 learned_addr learned my addr 172.21.15.72:0/3140031666 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.185+0000 7f2c45600700 1 -- 172.21.15.72:0/3140031666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c34009550 con 0x7f2c401076b0 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c45600700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:24.139 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c340039e0 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c34003b40 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4019bbe0 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c34024820 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.186+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4019c080 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.187+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4010ba90 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.187+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f2c3401ab80 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.188+0000 7f2c3e000700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 0x7f2c2803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.188+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2c3404cd00 con 0x7f2c401076b0 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.188+0000 7f2c44c00700 1 -- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 msgr2=0x7f2c2803b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:24.140 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.188+0000 7f2c44c00700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 0x7f2c2803b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.193+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c3401a950 con 0x7f2c401076b0 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.389+0000 7f2c44c00700 1 -- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 msgr2=0x7f2c2803b220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.389+0000 7f2c44c00700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 0x7f2c2803b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.471+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2c4019e960 con 0x7f2c401076b0 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.472+0000 7f2c3e000700 1 -- 172.21.15.72:0/3140031666 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f2c3402c350 con 0x7f2c401076b0 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.474+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 msgr2=0x7f2c2803b220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.474+0000 7f2c48000700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 0x7f2c2803b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 msgr2=0x7f2c4019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 shutdown_connections 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7f2c28038d60 0x7f2c2803b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:24.141 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 --2- 172.21.15.72:0/3140031666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c401076b0 0x7f2c4019b4a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:24.142 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 >> 172.21.15.72:0/3140031666 conn(0x7f2c40102c90 msgr2=0x7f2c401037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:24.142 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 shutdown_connections 2024-03-11T19:11:24.142 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:23.475+0000 7f2c48000700 1 -- 172.21.15.72:0/3140031666 wait complete. 2024-03-11T19:11:24.142 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-03-11T19:11:24.142 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 9... 2024-03-11T19:11:24.179 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:24.023+0000 7f1f8cbd9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:11:24.458 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:24.243+0000 7f1f8cbd9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:11:25.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:25.573+0000 7f1f8cbd9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:11:26.125 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:25.752+0000 7f1f8cbd9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:11:26.429 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:26.188+0000 7f1f8cbd9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:11:26.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:26.310+0000 7f1f8cbd9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:11:26.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:26.569+0000 7f1f8cbd9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:11:26.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:26.818+0000 7f1f8cbd9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:11:27.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:27.196+0000 7f1f8cbd9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:11:27.784 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:27.509+0000 7f1f8cbd9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:11:28.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:27.810+0000 7f1f8cbd9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:11:29.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:28.813+0000 7f1f8cbd9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:11:29.909 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:29.500+0000 7f1f8cbd9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:11:29.909 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:29.627+0000 7f1f8cbd9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:11:29.910 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:11:29.905+0000 7f1f8cbd9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:11:30.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: Active manager daemon x restarted 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: Activating manager daemon x 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: osdmap e3: 0 total, 0 up, 0 in 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: mgrmap e10: x(active, starting, since 0.00938314s) 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:11:30.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:29 smithi072 ceph-mon[31695]: Manager daemon x is now available 2024-03-11T19:11:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:11:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:11:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:30 smithi072 ceph-mon[31695]: mgrmap e11: x(active, since 1.01565s) 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.746+0000 7feb65800700 1 Processor -- start 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.746+0000 7feb65800700 1 -- start start 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.748+0000 7feb65800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.749+0000 7feb5fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.749+0000 7feb5fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49298/0 (socket says 172.21.15.72:49298) 2024-03-11T19:11:31.735 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.750+0000 7feb5fe00700 1 -- 172.21.15.72:0/2785898944 learned_addr learned my addr 172.21.15.72:0/2785898944 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.750+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60073900 con 0x7feb60072fa0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.751+0000 7feb5fe00700 1 -- 172.21.15.72:0/2785898944 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60073a40 con 0x7feb60072fa0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.752+0000 7feb5fe00700 1 --2- 172.21.15.72:0/2785898944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009b00 tx=0x7feb4c009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8083a66504361790 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.753+0000 7feb5ea00700 1 -- 172.21.15.72:0/2785898944 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb4c004160 con 0x7feb60072fa0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.753+0000 7feb5ea00700 1 -- 172.21.15.72:0/2785898944 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb4c0042c0 con 0x7feb60072fa0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.754+0000 7feb5ea00700 1 -- 172.21.15.72:0/2785898944 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb4c00ba20 con 0x7feb60072fa0 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.754+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 msgr2=0x7feb600733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.755+0000 7feb65800700 1 --2- 172.21.15.72:0/2785898944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009b00 tx=0x7feb4c009e10 comp rx=0 tx=0).stop 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.757+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 shutdown_connections 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.757+0000 7feb65800700 1 --2- 172.21.15.72:0/2785898944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb600733c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.757+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 >> 172.21.15.72:0/2785898944 conn(0x7feb6006e290 msgr2=0x7feb600706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.758+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 shutdown_connections 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.758+0000 7feb65800700 1 -- 172.21.15.72:0/2785898944 wait complete. 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.759+0000 7feb65800700 1 Processor -- start 2024-03-11T19:11:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.764+0000 7feb65800700 1 -- start start 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.765+0000 7feb65800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.765+0000 7feb65800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb601a20a0 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.767+0000 7feb5fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.767+0000 7feb5fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49302/0 (socket says 172.21.15.72:49302) 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.767+0000 7feb5fe00700 1 -- 172.21.15.72:0/1825023940 learned_addr learned my addr 172.21.15.72:0/1825023940 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.768+0000 7feb5fe00700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c009550 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.769+0000 7feb5fe00700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b770 tx=0x7feb4c003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.770+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb4c00b850 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.771+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb4c00be90 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.771+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb4c011420 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.771+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb601a22a0 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.771+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb601a2740 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.774+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7feb4c011580 con 0x7feb60072fa0 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.775+0000 7feb5cc00700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:31.737 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.775+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb4403b9c0 con 0x7feb44038df0 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.775+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feb4c04cf10 con 0x7feb60072fa0 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.775+0000 7feb5f400700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.775+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.976+0000 7feb5f400700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:24.976+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:25.377+0000 7feb5f400700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:25.377+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:26.178+0000 7feb5f400700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:26.178+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:27.780+0000 7feb5f400700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/2769969675 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:27.780+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:29.915+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7feb4c011b00 con 0x7feb60072fa0 2024-03-11T19:11:31.738 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:29.915+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 msgr2=0x7feb4403b2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:29.915+0000 7feb5cc00700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.918+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7feb4c04dfd0 con 0x7feb60072fa0 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.919+0000 7feb5cc00700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 0x7feb4403ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.919+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb4403b9c0 con 0x7feb4403c870 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.922+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 0x7feb4403ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.924+0000 7feb5f400700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 0x7feb4403ec60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb58003d90 tx=0x7feb58007440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.926+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7feb4403b9c0 con 0x7feb4403c870 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7feb60068d60 con 0x7feb4403c870 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb5cc00700 1 -- 172.21.15.72:0/1825023940 <== mgr.14152 v2:172.21.15.72:6800/537254334 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7feb60068d60 con 0x7feb4403c870 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 msgr2=0x7feb4403ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 0x7feb4403ec60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb58003d90 tx=0x7feb58007440 comp rx=0 tx=0).stop 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 msgr2=0x7feb601a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00b770 tx=0x7feb4c003980 comp rx=0 tx=0).stop 2024-03-11T19:11:31.739 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 shutdown_connections 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7feb4403c870 0x7feb4403ec60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb60072fa0 0x7feb601a1b60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 --2- 172.21.15.72:0/1825023940 >> [v2:172.21.15.72:6800/2769969675,v1:172.21.15.72:6801/2769969675] conn(0x7feb44038df0 0x7feb4403b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.940+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 >> 172.21.15.72:0/1825023940 conn(0x7feb6006e290 msgr2=0x7feb6006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.941+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 shutdown_connections 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:30.941+0000 7feb65800700 1 -- 172.21.15.72:0/1825023940 wait complete. 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 9 is available 2024-03-11T19:11:31.740 INFO:teuthology.orchestra.run.smithi072.stdout:Generating a dashboard self-signed certificate... 2024-03-11T19:11:32.245 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: [11/Mar/2024:19:11:30] ENGINE Bus STARTING 2024-03-11T19:11:32.245 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: [11/Mar/2024:19:11:30] ENGINE Serving on https://172.21.15.72:7150 2024-03-11T19:11:32.245 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:11:32.245 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:11:32.246 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: [11/Mar/2024:19:11:31] ENGINE Serving on http://172.21.15.72:8765 2024-03-11T19:11:32.246 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:31 smithi072 ceph-mon[31695]: [11/Mar/2024:19:11:31] ENGINE Bus STARTED 2024-03-11T19:11:33.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:32 smithi072 ceph-mon[31695]: mgrmap e12: x(active, since 2s) 2024-03-11T19:11:33.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:32 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:33.885 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.756+0000 7f8646600700 1 Processor -- start 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.756+0000 7f8646600700 1 -- start start 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.756+0000 7f8646600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.756+0000 7f8646600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401080a0 con 0x7f86401076b0 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.757+0000 7f863f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.757+0000 7f863f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34080/0 (socket says 172.21.15.72:34080) 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.757+0000 7f863f400700 1 -- 172.21.15.72:0/71731601 learned_addr learned my addr 172.21.15.72:0/71731601 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.757+0000 7f863f400700 1 -- 172.21.15.72:0/71731601 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640108860 con 0x7f86401076b0 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.758+0000 7f863f400700 1 --2- 172.21.15.72:0/71731601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86300098a0 tx=0x7f8630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be123332419a0c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.758+0000 7f863e000700 1 -- 172.21.15.72:0/71731601 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8630004160 con 0x7f86401076b0 2024-03-11T19:11:33.886 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.758+0000 7f863e000700 1 -- 172.21.15.72:0/71731601 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86300042c0 con 0x7f86401076b0 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.758+0000 7f863e000700 1 -- 172.21.15.72:0/71731601 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8630011a80 con 0x7f86401076b0 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 -- 172.21.15.72:0/71731601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 msgr2=0x7f8640107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 --2- 172.21.15.72:0/71731601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86300098a0 tx=0x7f8630009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 -- 172.21.15.72:0/71731601 shutdown_connections 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 --2- 172.21.15.72:0/71731601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f8640107ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 -- 172.21.15.72:0/71731601 >> 172.21.15.72:0/71731601 conn(0x7f8640102cd0 msgr2=0x7f86401050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 -- 172.21.15.72:0/71731601 shutdown_connections 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.759+0000 7f8646600700 1 -- 172.21.15.72:0/71731601 wait complete. 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.760+0000 7f8646600700 1 Processor -- start 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.760+0000 7f8646600700 1 -- start start 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f8646600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f8646600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864019b9e0 con 0x7f86401076b0 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f863f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f863f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34086/0 (socket says 172.21.15.72:34086) 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f863f400700 1 -- 172.21.15.72:0/1566729436 learned_addr learned my addr 172.21.15.72:0/1566729436 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.761+0000 7f863f400700 1 -- 172.21.15.72:0/1566729436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009550 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f863f400700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8630003f20 tx=0x7f8630011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86300038b0 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8630003a10 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864019bbe0 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f863001a5c0 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.762+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864019c080 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.763+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f8630024430 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.763+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864010ba90 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.764+0000 7f8637e00700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 0x7f862403b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.764+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f863004cf60 con 0x7f86401076b0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.764+0000 7f863ea00700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 0x7f862403b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:33.888 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.764+0000 7f863ea00700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 0x7f862403b1d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f862c006fd0 tx=0x7f862c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:33.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.769+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8630018530 con 0x7f86401076b0 2024-03-11T19:11:33.889 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:32.996+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f8640103f40 con 0x7f8624038d10 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.210+0000 7f8637e00700 1 -- 172.21.15.72:0/1566729436 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f8640103f40 con 0x7f8624038d10 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 msgr2=0x7f862403b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 0x7f862403b1d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f862c006fd0 tx=0x7f862c006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 msgr2=0x7f864019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8630003f20 tx=0x7f8630011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 shutdown_connections 2024-03-11T19:11:33.890 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8624038d10 0x7f862403b1d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:33.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 --2- 172.21.15.72:0/1566729436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86401076b0 0x7f864019b4a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:33.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 >> 172.21.15.72:0/1566729436 conn(0x7f8640102cd0 msgr2=0x7f8640103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:33.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 shutdown_connections 2024-03-11T19:11:33.891 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:33.213+0000 7f8646600700 1 -- 172.21.15.72:0/1566729436 wait complete. 2024-03-11T19:11:33.891 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial admin user... 2024-03-11T19:11:34.172 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:33 smithi072 ceph-mon[31695]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:34.172 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:33 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:34.172 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:33 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:35.597 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VVHYGtWssHKvJvSKZS92du/jbkN6HaGkUdm19Isi8EbkatOKk2h9O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710184294, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-11T19:11:35.597 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.453+0000 7f430ac00700 1 Processor -- start 2024-03-11T19:11:35.597 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.453+0000 7f430ac00700 1 -- start start 2024-03-11T19:11:35.597 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.453+0000 7f430ac00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.453+0000 7f430ac00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430407b830 con 0x7f430407ce40 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.454+0000 7f4303e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.454+0000 7f4303e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34100/0 (socket says 172.21.15.72:34100) 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.454+0000 7f4303e00700 1 -- 172.21.15.72:0/3286037081 learned_addr learned my addr 172.21.15.72:0/3286037081 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.454+0000 7f4303e00700 1 -- 172.21.15.72:0/3286037081 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430407b970 con 0x7f430407ce40 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.454+0000 7f4303e00700 1 --2- 172.21.15.72:0/3286037081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098a0 tx=0x7f42f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57344db2f8e26eca server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.455+0000 7f4302a00700 1 -- 172.21.15.72:0/3286037081 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42f4004160 con 0x7f430407ce40 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.455+0000 7f4302a00700 1 -- 172.21.15.72:0/3286037081 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f40042c0 con 0x7f430407ce40 2024-03-11T19:11:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.455+0000 7f4302a00700 1 -- 172.21.15.72:0/3286037081 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42f4011a80 con 0x7f430407ce40 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.455+0000 7f430ac00700 1 -- 172.21.15.72:0/3286037081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 msgr2=0x7f430407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.455+0000 7f430ac00700 1 --2- 172.21.15.72:0/3286037081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098a0 tx=0x7f42f4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.456+0000 7f430ac00700 1 -- 172.21.15.72:0/3286037081 shutdown_connections 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.456+0000 7f430ac00700 1 --2- 172.21.15.72:0/3286037081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430407b2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.456+0000 7f430ac00700 1 -- 172.21.15.72:0/3286037081 >> 172.21.15.72:0/3286037081 conn(0x7f4304102d00 msgr2=0x7f4304105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.456+0000 7f430ac00700 1 -- 172.21.15.72:0/3286037081 shutdown_connections 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.456+0000 7f430ac00700 1 -- 172.21.15.72:0/3286037081 wait complete. 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.457+0000 7f430ac00700 1 Processor -- start 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.457+0000 7f430ac00700 1 -- start start 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.457+0000 7f430ac00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.457+0000 7f430ac00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430419db20 con 0x7f430407ce40 2024-03-11T19:11:35.599 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.458+0000 7f4303e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.458+0000 7f4303e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34108/0 (socket says 172.21.15.72:34108) 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.458+0000 7f4303e00700 1 -- 172.21.15.72:0/1766564834 learned_addr learned my addr 172.21.15.72:0/1766564834 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.458+0000 7f4303e00700 1 -- 172.21.15.72:0/1766564834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4009550 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.458+0000 7f4303e00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42f4003f20 tx=0x7f42f4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.459+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42f4003a70 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.459+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f4003bd0 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.459+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430419dd20 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.459+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42f401a800 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.459+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430419e1c0 con 0x7f430407ce40 2024-03-11T19:11:35.600 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.460+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f42f4024430 con 0x7f430407ce40 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.460+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430410daa0 con 0x7f430407ce40 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.460+0000 7f4300c00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 0x7f42e803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.461+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42f404d1b0 con 0x7f430407ce40 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.461+0000 7f4303400700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 0x7f42e803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.462+0000 7f4303400700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 0x7f42e803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006fd0 tx=0x7f42f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.466+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42f4024aa0 con 0x7f430407ce40 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.713+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f43040630c0 con 0x7f42e8038cb0 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.948+0000 7f4300c00700 1 -- 172.21.15.72:0/1766564834 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f43040630c0 con 0x7f42e8038cb0 2024-03-11T19:11:35.601 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 msgr2=0x7f42e803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 0x7f42e803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006fd0 tx=0x7f42f0006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 msgr2=0x7f430419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42f4003f20 tx=0x7f42f4011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 shutdown_connections 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42e8038cb0 0x7f42e803b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 --2- 172.21.15.72:0/1766564834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f430407ce40 0x7f430419d5e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 >> 172.21.15.72:0/1766564834 conn(0x7f4304102d00 msgr2=0x7f43041037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.951+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 shutdown_connections 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:34.952+0000 7f430ac00700 1 -- 172.21.15.72:0/1766564834 wait complete. 2024-03-11T19:11:35.602 INFO:teuthology.orchestra.run.smithi072.stdout:Fetching dashboard port number... 2024-03-11T19:11:35.912 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:35 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:35.912 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:35 smithi072 ceph-mon[31695]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:35.912 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:35 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:36.791 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/35605541' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-11T19:11:37.472 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 8443 2024-03-11T19:11:37.472 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.221+0000 7f99b0e00700 1 Processor -- start 2024-03-11T19:11:37.472 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.221+0000 7f99b0e00700 1 -- start start 2024-03-11T19:11:37.472 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99b0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:37.472 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99b0e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac10a300 con 0x7f99ac109910 2024-03-11T19:11:37.473 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99a9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:37.473 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99a9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34120/0 (socket says 172.21.15.72:34120) 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99a9e00700 1 -- 172.21.15.72:0/2389094295 learned_addr learned my addr 172.21.15.72:0/2389094295 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.222+0000 7f99a9e00700 1 -- 172.21.15.72:0/2389094295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac10aa70 con 0x7f99ac109910 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.223+0000 7f99a9e00700 1 --2- 172.21.15.72:0/2389094295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf32c5f83ffe34a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.223+0000 7f99a8a00700 1 -- 172.21.15.72:0/2389094295 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a4004160 con 0x7f99ac109910 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.223+0000 7f99a8a00700 1 -- 172.21.15.72:0/2389094295 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99a40042c0 con 0x7f99ac109910 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.223+0000 7f99a8a00700 1 -- 172.21.15.72:0/2389094295 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a4011a80 con 0x7f99ac109910 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 -- 172.21.15.72:0/2389094295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 msgr2=0x7f99ac109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 --2- 172.21.15.72:0/2389094295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 -- 172.21.15.72:0/2389094295 shutdown_connections 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 --2- 172.21.15.72:0/2389094295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac109d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:37.475 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 -- 172.21.15.72:0/2389094295 >> 172.21.15.72:0/2389094295 conn(0x7f99ac07ce20 msgr2=0x7f99ac1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:37.476 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 -- 172.21.15.72:0/2389094295 shutdown_connections 2024-03-11T19:11:37.476 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.224+0000 7f99b0e00700 1 -- 172.21.15.72:0/2389094295 wait complete. 2024-03-11T19:11:37.476 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.225+0000 7f99b0e00700 1 Processor -- start 2024-03-11T19:11:37.476 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.225+0000 7f99b0e00700 1 -- start start 2024-03-11T19:11:37.476 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.225+0000 7f99b0e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:37.478 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.225+0000 7f99b0e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19dbb0 con 0x7f99ac109910 2024-03-11T19:11:37.478 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:37.478 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34136/0 (socket says 172.21.15.72:34136) 2024-03-11T19:11:37.478 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a9e00700 1 -- 172.21.15.72:0/35605541 learned_addr learned my addr 172.21.15.72:0/35605541 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a9e00700 1 -- 172.21.15.72:0/35605541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009550 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a9e00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.226+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a40039e0 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.227+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99a4003b40 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.227+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99a4024820 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.227+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac19ddb0 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.227+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac19e250 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.228+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f99a401a430 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.228+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac10dca0 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.228+0000 7f99a2a00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 0x7f999003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.228+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99a404ce10 con 0x7f99ac109910 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.229+0000 7f99a9400700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 0x7f999003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.229+0000 7f99a9400700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 0x7f999003b270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:37.479 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.234+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99a401ab80 con 0x7f99ac109910 2024-03-11T19:11:37.480 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.458+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f99ac1a0b30 con 0x7f99ac109910 2024-03-11T19:11:37.480 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.458+0000 7f99a2a00700 1 -- 172.21.15.72:0/35605541 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f99a4019e60 con 0x7f99ac109910 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 msgr2=0x7f999003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 0x7f999003b270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 msgr2=0x7f99ac19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 shutdown_connections 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9990038db0 0x7f999003b270 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 --2- 172.21.15.72:0/35605541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99ac109910 0x7f99ac19d670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 >> 172.21.15.72:0/35605541 conn(0x7f99ac07ce20 msgr2=0x7f99ac106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 shutdown_connections 2024-03-11T19:11:37.481 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:36.461+0000 7f99b0e00700 1 -- 172.21.15.72:0/35605541 wait complete. 2024-03-11T19:11:37.517 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:11:37.518 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-03-11T19:11:37.537 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:11:37.538 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-03-11T19:11:37.538 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-03-11T19:11:37.538 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph Dashboard is now available at: 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: URL: https://smithi072.front.sepia.ceph.com:8443/ 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: User: admin 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: Password: flyiofcmin 2024-03-11T19:11:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:37.543 INFO:teuthology.orchestra.run.smithi072.stdout:Saving cluster configuration to /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config directory 2024-03-11T19:11:37.543 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling autotune for osd_memory_target 2024-03-11T19:11:37.880 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:37 smithi072 ceph-mon[31695]: mgrmap e13: x(active, since 6s) 2024-03-11T19:11:39.439 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.557+0000 7f182a000700 1 Processor -- start 2024-03-11T19:11:39.439 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.557+0000 7f182a000700 1 -- start start 2024-03-11T19:11:39.439 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.558+0000 7f182a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:39.439 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.558+0000 7f182a000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182410eab0 con 0x7f182410e0c0 2024-03-11T19:11:39.440 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.558+0000 7f1823400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.558+0000 7f1823400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34140/0 (socket says 172.21.15.72:34140) 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.558+0000 7f1823400700 1 -- 172.21.15.72:0/2085238225 learned_addr learned my addr 172.21.15.72:0/2085238225 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.559+0000 7f1823400700 1 -- 172.21.15.72:0/2085238225 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182410ee60 con 0x7f182410e0c0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.559+0000 7f1823400700 1 --2- 172.21.15.72:0/2085238225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb6ecd15213f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.559+0000 7f1822000700 1 -- 172.21.15.72:0/2085238225 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814004160 con 0x7f182410e0c0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.559+0000 7f1822000700 1 -- 172.21.15.72:0/2085238225 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18140042c0 con 0x7f182410e0c0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.559+0000 7f1822000700 1 -- 172.21.15.72:0/2085238225 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814011a80 con 0x7f182410e0c0 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 -- 172.21.15.72:0/2085238225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 msgr2=0x7f182410e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:39.441 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 --2- 172.21.15.72:0/2085238225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:39.442 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 -- 172.21.15.72:0/2085238225 shutdown_connections 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 --2- 172.21.15.72:0/2085238225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f182410e4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 -- 172.21.15.72:0/2085238225 >> 172.21.15.72:0/2085238225 conn(0x7f182407cd90 msgr2=0x7f182407f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 -- 172.21.15.72:0/2085238225 shutdown_connections 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.560+0000 7f182a000700 1 -- 172.21.15.72:0/2085238225 wait complete. 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.561+0000 7f182a000700 1 Processor -- start 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.561+0000 7f182a000700 1 -- start start 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.561+0000 7f182a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f182a000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18241a1f10 con 0x7f182410e0c0 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f1823400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f1823400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34142/0 (socket says 172.21.15.72:34142) 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f1823400700 1 -- 172.21.15.72:0/3036479320 learned_addr learned my addr 172.21.15.72:0/3036479320 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:39.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f1823400700 1 -- 172.21.15.72:0/3036479320 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009550 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.562+0000 7f1823400700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1814011740 tx=0x7f1814011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.563+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814003b20 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.563+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f181401a910 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.563+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18140248c0 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.563+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18241a2110 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.563+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18241a25b0 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.564+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f181401aa80 con 0x7f182410e0c0 2024-03-11T19:11:39.445 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.564+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18241120b0 con 0x7f182410e0c0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.564+0000 7f181be00700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 0x7f180803b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.565+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f181404d320 con 0x7f182410e0c0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.565+0000 7f1822a00700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 0x7f180803b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.565+0000 7f1822a00700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 0x7f180803b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.570+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f181401adf0 con 0x7f182410e0c0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.786+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f18241a4e90 con 0x7f182410e0c0 2024-03-11T19:11:39.447 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.787+0000 7f181be00700 1 -- 172.21.15.72:0/3036479320 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f181404cbb0 con 0x7f182410e0c0 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 msgr2=0x7f180803b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 0x7f180803b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 msgr2=0x7f18241a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1814011740 tx=0x7f1814011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 shutdown_connections 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1808038da0 0x7f180803b260 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1810006fd0 tx=0x7f1810006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 --2- 172.21.15.72:0/3036479320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f182410e0c0 0x7f18241a19d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 >> 172.21.15.72:0/3036479320 conn(0x7f182407cd90 msgr2=0x7f182407d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.790+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 shutdown_connections 2024-03-11T19:11:39.448 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:38.791+0000 7f182a000700 1 -- 172.21.15.72:0/3036479320 wait complete. 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.035+0000 7f74b1a00700 1 Processor -- start 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.036+0000 7f74b1a00700 1 -- start start 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.036+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.036+0000 7f74b1a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac10a2e0 con 0x7f74ac1098f0 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.037+0000 7f74aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.037+0000 7f74aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34152/0 (socket says 172.21.15.72:34152) 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.037+0000 7f74aaa00700 1 -- 172.21.15.72:0/3414634079 learned_addr learned my addr 172.21.15.72:0/3414634079 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.037+0000 7f74aaa00700 1 -- 172.21.15.72:0/3414634079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac10aa50 con 0x7f74ac1098f0 2024-03-11T19:11:41.114 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.038+0000 7f74aaa00700 1 --2- 172.21.15.72:0/3414634079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098a0 tx=0x7f749c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d322ababc5e20cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.038+0000 7f74aa000700 1 -- 172.21.15.72:0/3414634079 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f749c004160 con 0x7f74ac1098f0 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.038+0000 7f74aa000700 1 -- 172.21.15.72:0/3414634079 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f749c0042c0 con 0x7f74ac1098f0 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.038+0000 7f74aa000700 1 -- 172.21.15.72:0/3414634079 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f749c011a80 con 0x7f74ac1098f0 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 -- 172.21.15.72:0/3414634079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 msgr2=0x7f74ac109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 --2- 172.21.15.72:0/3414634079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098a0 tx=0x7f749c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 -- 172.21.15.72:0/3414634079 shutdown_connections 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 --2- 172.21.15.72:0/3414634079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac109d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 -- 172.21.15.72:0/3414634079 >> 172.21.15.72:0/3414634079 conn(0x7f74ac07ce20 msgr2=0x7f74ac1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 -- 172.21.15.72:0/3414634079 shutdown_connections 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.039+0000 7f74b1a00700 1 -- 172.21.15.72:0/3414634079 wait complete. 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.040+0000 7f74b1a00700 1 Processor -- start 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.040+0000 7f74b1a00700 1 -- start start 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74b1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74b1a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac19db10 con 0x7f74ac1098f0 2024-03-11T19:11:41.115 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34156/0 (socket says 172.21.15.72:34156) 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74aaa00700 1 -- 172.21.15.72:0/4095903325 learned_addr learned my addr 172.21.15.72:0/4095903325 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.041+0000 7f74aaa00700 1 -- 172.21.15.72:0/4095903325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009550 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74aaa00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f749c011740 tx=0x7f749c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f749c0039e0 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f749c003b40 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f749c024710 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac19dd10 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.042+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac19e1b0 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.043+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f749c01a430 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.043+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac10dc80 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.044+0000 7f74a3e00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 0x7f749003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.044+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f749c04d0b0 con 0x7f74ac1098f0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.044+0000 7f74a2000700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 0x7f749003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.045+0000 7f74a2000700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 0x7f749003b270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7494006fd0 tx=0x7f7494006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:41.116 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.049+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f749c018530 con 0x7f74ac1098f0 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.384+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f74ac1a0a90 con 0x7f74ac1098f0 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.389+0000 7f74a3e00700 1 -- 172.21.15.72:0/4095903325 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f749c018350 con 0x7f74ac1098f0 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 msgr2=0x7f749003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 0x7f749003b270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7494006fd0 tx=0x7f7494006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 msgr2=0x7f74ac19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f749c011740 tx=0x7f749c011b00 comp rx=0 tx=0).stop 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 shutdown_connections 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f7490038db0 0x7f749003b270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 --2- 172.21.15.72:0/4095903325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74ac1098f0 0x7f74ac19d5d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:41.117 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 >> 172.21.15.72:0/4095903325 conn(0x7f74ac07ce20 msgr2=0x7f74ac106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.392+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 shutdown_connections 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-03-11T19:11:40.393+0000 7f74b1a00700 1 -- 172.21.15.72:0/4095903325 wait complete. 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:11:41.118 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout:Or, if you are only running a single cluster on this host: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: ceph telemetry on 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout:For more information see: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:11:41.119 INFO:teuthology.orchestra.run.smithi072.stdout:Bootstrap complete. 2024-03-11T19:11:41.208 INFO:tasks.cephadm:Fetching config... 2024-03-11T19:11:41.208 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:11:41.208 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-11T19:11:41.258 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-11T19:11:41.258 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:11:41.258 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-11T19:11:41.297 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-11T19:11:41.297 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:11:41.297 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-11T19:11:41.416 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-11T19:11:41.416 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:11:41.416 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-11T19:11:41.510 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-11T19:11:41.510 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:11:41.638 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:11:41.672 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:41 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4095903325' entity='client.admin' 2024-03-11T19:11:41.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:11:41.736 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:11:41.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBbskmaSp7xSzAMR6cKIiuCZAmF1JY/34TkEGxttSywS0x3rBHW8HaMPoW1ijAB+TqvvRRPc/r6UhLoU6Z63UT9Wq/CANM2207bz787Fa7m3UeH9DfyjOTPSgETCeMRKSMW7VQNttQZArYrjdmuHDhI2CrzucQIGiFyTZ8UCJ5wm7AZawZimMS/85AbakFTfJ6nkkGs4/Ecv+T5JoGMNnv7tw3ViB2yqYp612csR/rWFiNZ0eEkCbdfnXsQKBlhV3sLdmfqeA9NidVXfqoOZ1qL7Lm+h5qNaPEXJ6fsQnobwPibCsbMJLQ2ebn8wCmkiHF16EwJtD5dGMxN4GcLMBP4I2XQWx17Fb+7m7gz+SG4vxG+toFy+nMSdyLAnWfrria8VcRz0OT1h16IwOpC5nr4wKv42rpwGRRmElgzlQgdZuQi7UtxOk4IUjVuGKsk4RzMReraFpjqDGr5DQVTmzseZDJPdpZyw1w6RHPWn6GuytDpy+ZBqgVMcdquKLZb60= ceph-d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:11:41.812 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-11T19:11:42.221 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.813+0000 7fe53e000700 1 -- 172.21.15.72:0/1463281193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 msgr2=0x7fe538101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.813+0000 7fe53e000700 1 --2- 172.21.15.72:0/1463281193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538101af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe528009910 tx=0x7fe528009c20 comp rx=0 tx=0).stop 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.814+0000 7fe53e000700 1 -- 172.21.15.72:0/1463281193 shutdown_connections 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.814+0000 7fe53e000700 1 --2- 172.21.15.72:0/1463281193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538101af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.814+0000 7fe53e000700 1 -- 172.21.15.72:0/1463281193 >> 172.21.15.72:0/1463281193 conn(0x7fe5380fcc90 msgr2=0x7fe5380ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.814+0000 7fe53e000700 1 -- 172.21.15.72:0/1463281193 shutdown_connections 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.814+0000 7fe53e000700 1 -- 172.21.15.72:0/1463281193 wait complete. 2024-03-11T19:11:44.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.815+0000 7fe53e000700 1 Processor -- start 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.815+0000 7fe53e000700 1 -- start start 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe53e000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe53e000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381970f0 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe537400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe537400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45268/0 (socket says 172.21.15.72:45268) 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe537400700 1 -- 172.21.15.72:0/1691115373 learned_addr learned my addr 172.21.15.72:0/1691115373 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.816+0000 7fe537400700 1 -- 172.21.15.72:0/1691115373 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280095f0 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe537400700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe528004f40 tx=0x7fe528005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe52801d070 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5280044a0 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe528003a40 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5381972f0 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.817+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe538197790 con 0x7fe5381016d0 2024-03-11T19:11:44.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.818+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe538190530 con 0x7fe5381016d0 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.819+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fe528003ba0 con 0x7fe5381016d0 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.819+0000 7fe52fe00700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 0x7fe51c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.820+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe52804c860 con 0x7fe5381016d0 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.820+0000 7fe536a00700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 0x7fe51c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.821+0000 7fe536a00700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 0x7fe51c03b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:44.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:44.825+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe528029540 con 0x7fe5381016d0 2024-03-11T19:11:45.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.050+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fe5380632a0 con 0x7fe5381016d0 2024-03-11T19:11:45.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.054+0000 7fe52fe00700 1 -- 172.21.15.72:0/1691115373 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe528026030 con 0x7fe5381016d0 2024-03-11T19:11:45.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 msgr2=0x7fe51c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 0x7fe51c03b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 msgr2=0x7fe538196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe528004f40 tx=0x7fe528005020 comp rx=0 tx=0).stop 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 shutdown_connections 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe51c038db0 0x7fe51c03b270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 --2- 172.21.15.72:0/1691115373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5381016d0 0x7fe538196bb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 >> 172.21.15.72:0/1691115373 conn(0x7fe5380fcc90 msgr2=0x7fe5380fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:45.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 shutdown_connections 2024-03-11T19:11:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:45.058+0000 7fe53e000700 1 -- 172.21.15.72:0/1691115373 wait complete. 2024-03-11T19:11:45.756 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-11T19:11:45.756 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-11T19:11:46.179 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:11:46.328 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1691115373' entity='client.admin' 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:48.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 -- 172.21.15.72:0/4011510350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 msgr2=0x7f6980074100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:48.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 --2- 172.21.15.72:0/4011510350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f6980074100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6970009910 tx=0x7f6970009c20 comp rx=0 tx=0).stop 2024-03-11T19:11:48.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 -- 172.21.15.72:0/4011510350 shutdown_connections 2024-03-11T19:11:48.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 --2- 172.21.15.72:0/4011510350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f6980074100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:48.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 -- 172.21.15.72:0/4011510350 >> 172.21.15.72:0/4011510350 conn(0x7f69800fcab0 msgr2=0x7f69800feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:48.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 -- 172.21.15.72:0/4011510350 shutdown_connections 2024-03-11T19:11:48.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.597+0000 7f6985a00700 1 -- 172.21.15.72:0/4011510350 wait complete. 2024-03-11T19:11:48.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.599+0000 7f6985a00700 1 Processor -- start 2024-03-11T19:11:48.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.599+0000 7f6985a00700 1 -- start start 2024-03-11T19:11:48.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.599+0000 7f6985a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:48.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.599+0000 7f6985a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698019d560 con 0x7f6980075ca0 2024-03-11T19:11:48.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:48.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45290/0 (socket says 172.21.15.72:45290) 2024-03-11T19:11:48.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697fe00700 1 -- 172.21.15.72:0/501747842 learned_addr learned my addr 172.21.15.72:0/501747842 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:48.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697fe00700 1 -- 172.21.15.72:0/501747842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700095f0 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697fe00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6970004f40 tx=0x7f6970005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.600+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f697001d070 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.601+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69700044a0 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.601+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698019d760 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.601+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6970003a40 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.601+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698019dc00 con 0x7f6980075ca0 2024-03-11T19:11:48.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.602+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f6970003cb0 con 0x7f6980075ca0 2024-03-11T19:11:48.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.602+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6980196960 con 0x7f6980075ca0 2024-03-11T19:11:48.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.602+0000 7f697cc00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 0x7f696403b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:48.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.602+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f697004cc40 con 0x7f6980075ca0 2024-03-11T19:11:48.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.603+0000 7f697f400700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 0x7f696403b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:48.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.603+0000 7f697f400700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 0x7f696403b270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f696c006fd0 tx=0x7f696c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:48.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.608+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6970029360 con 0x7f6980075ca0 2024-03-11T19:11:48.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.831+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f69800620b0 con 0x7f6964038db0 2024-03-11T19:11:48.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.839+0000 7f697cc00700 1 -- 172.21.15.72:0/501747842 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f69800620b0 con 0x7f6964038db0 2024-03-11T19:11:48.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 msgr2=0x7f696403b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:48.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 0x7f696403b270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f696c006fd0 tx=0x7f696c006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 msgr2=0x7f698019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6970004f40 tx=0x7f6970005020 comp rx=0 tx=0).stop 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 shutdown_connections 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6964038db0 0x7f696403b270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 --2- 172.21.15.72:0/501747842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6980075ca0 0x7f698019d020 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.842+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 >> 172.21.15.72:0/501747842 conn(0x7f69800fcab0 msgr2=0x7f69800fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.843+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 shutdown_connections 2024-03-11T19:11:48.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:48.843+0000 7f6985a00700 1 -- 172.21.15.72:0/501747842 wait complete. 2024-03-11T19:11:49.579 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-03-11T19:11:49.580 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:11:49.580 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:11:49.597 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:11:49.598 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:11:49.655 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-03-11T19:11:49.655 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch host add smithi106 2024-03-11T19:11:50.063 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:11:50.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:49 smithi072 ceph-mon[31695]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:11:51.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:50 smithi072 ceph-mon[31695]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:11:51.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:50 smithi072 ceph-mon[31695]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:51.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:50 smithi072 ceph-mon[31695]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 -- 172.21.15.72:0/1995384025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 msgr2=0x7efd38109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 --2- 172.21.15.72:0/1995384025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38109c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd28009910 tx=0x7efd28009c20 comp rx=0 tx=0).stop 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 -- 172.21.15.72:0/1995384025 shutdown_connections 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 --2- 172.21.15.72:0/1995384025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38109c50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 -- 172.21.15.72:0/1995384025 >> 172.21.15.72:0/1995384025 conn(0x7efd380ff360 msgr2=0x7efd38101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 -- 172.21.15.72:0/1995384025 shutdown_connections 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.076+0000 7efd3d400700 1 -- 172.21.15.72:0/1995384025 wait complete. 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.077+0000 7efd3d400700 1 Processor -- start 2024-03-11T19:11:52.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.077+0000 7efd3d400700 1 -- start start 2024-03-11T19:11:52.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.078+0000 7efd3d400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:52.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.078+0000 7efd3d400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38199760 con 0x7efd38109870 2024-03-11T19:11:52.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.078+0000 7efd36800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:52.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.078+0000 7efd36800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46520/0 (socket says 172.21.15.72:46520) 2024-03-11T19:11:52.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.078+0000 7efd36800700 1 -- 172.21.15.72:0/2866720223 learned_addr learned my addr 172.21.15.72:0/2866720223 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd36800700 1 -- 172.21.15.72:0/2866720223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280095f0 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd36800700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd28006010 tx=0x7efd28004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd2801d070 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2800bbd0 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd381999c0 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd28003880 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.079+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd38199e40 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.080+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7efd2800bd40 con 0x7efd38109870 2024-03-11T19:11:52.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.080+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380632a0 con 0x7efd38109870 2024-03-11T19:11:52.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.081+0000 7efd2f400700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 0x7efd1c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:11:52.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.081+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd2804ca30 con 0x7efd38109870 2024-03-11T19:11:52.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.081+0000 7efd35e00700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 0x7efd1c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:11:52.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.082+0000 7efd35e00700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 0x7efd1c03b270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:11:52.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.086+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd2802ae20 con 0x7efd38109870 2024-03-11T19:11:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:51 smithi072 ceph-mon[31695]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:11:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:52.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:52.313+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}) v1 -- 0x7efd3810ddf0 con 0x7efd1c038db0 2024-03-11T19:11:53.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:52 smithi072 ceph-mon[31695]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:54.104 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:53 smithi072 ceph-mon[31695]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:11:54.104 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:53 smithi072 ceph-mon[31695]: Deploying cephadm binary to smithi106 2024-03-11T19:11:55.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:54 smithi072 ceph-mon[31695]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:57.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.094+0000 7efd2f400700 1 -- 172.21.15.72:0/2866720223 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efd3810ddf0 con 0x7efd1c038db0 2024-03-11T19:11:57.099 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-03-11T19:11:57.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 msgr2=0x7efd1c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:57.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 0x7efd1c03b270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24006e40 comp rx=0 tx=0).stop 2024-03-11T19:11:57.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 msgr2=0x7efd38199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:11:57.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd28006010 tx=0x7efd28004ff0 comp rx=0 tx=0).stop 2024-03-11T19:11:57.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 shutdown_connections 2024-03-11T19:11:57.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd1c038db0 0x7efd1c03b270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:57.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 --2- 172.21.15.72:0/2866720223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd38109870 0x7efd38199220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:11:57.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 >> 172.21.15.72:0/2866720223 conn(0x7efd380ff360 msgr2=0x7efd38100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:11:57.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.098+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 shutdown_connections 2024-03-11T19:11:57.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:11:57.099+0000 7efd3d400700 1 -- 172.21.15.72:0/2866720223 wait complete. 2024-03-11T19:11:57.405 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:56 smithi072 ceph-mon[31695]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:57.855 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:11:58.288 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:11:58.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:58 smithi072 ceph-mon[31695]: Added host smithi106 2024-03-11T19:11:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:11:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:11:59.452 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:59 smithi072 ceph-mon[31695]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:11:59.452 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:11:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:00.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.308+0000 7f5a71400700 1 -- 172.21.15.72:0/831579376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 msgr2=0x7f5a6c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.308+0000 7f5a71400700 1 --2- 172.21.15.72:0/831579376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c109c50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009910 tx=0x7f5a5c009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.309+0000 7f5a71400700 1 -- 172.21.15.72:0/831579376 shutdown_connections 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.309+0000 7f5a71400700 1 --2- 172.21.15.72:0/831579376 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c109c50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.309+0000 7f5a71400700 1 -- 172.21.15.72:0/831579376 >> 172.21.15.72:0/831579376 conn(0x7f5a6c0ff360 msgr2=0x7f5a6c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.309+0000 7f5a71400700 1 -- 172.21.15.72:0/831579376 shutdown_connections 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.309+0000 7f5a71400700 1 -- 172.21.15.72:0/831579376 wait complete. 2024-03-11T19:12:00.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.310+0000 7f5a71400700 1 Processor -- start 2024-03-11T19:12:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.310+0000 7f5a71400700 1 -- start start 2024-03-11T19:12:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.310+0000 7f5a71400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.310+0000 7f5a71400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c199720 con 0x7f5a6c109870 2024-03-11T19:12:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a6a800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:00.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a6a800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46538/0 (socket says 172.21.15.72:46538) 2024-03-11T19:12:00.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a6a800700 1 -- 172.21.15.72:0/46312941 learned_addr learned my addr 172.21.15.72:0/46312941 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:00.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a6a800700 1 -- 172.21.15.72:0/46312941 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0095f0 con 0x7f5a6c109870 2024-03-11T19:12:00.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a6a800700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c004f40 tx=0x7f5a5c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:00.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c01d070 con 0x7f5a6c109870 2024-03-11T19:12:00.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.311+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5c00bbd0 con 0x7f5a6c109870 2024-03-11T19:12:00.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.312+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c003900 con 0x7f5a6c109870 2024-03-11T19:12:00.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.312+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c199980 con 0x7f5a6c109870 2024-03-11T19:12:00.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.312+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c199e00 con 0x7f5a6c109870 2024-03-11T19:12:00.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.313+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f5a5c00bd40 con 0x7f5a6c109870 2024-03-11T19:12:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.313+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c0632a0 con 0x7f5a6c109870 2024-03-11T19:12:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.313+0000 7f5a63400700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 0x7f5a5003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.313+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a5c04cbb0 con 0x7f5a6c109870 2024-03-11T19:12:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.313+0000 7f5a69e00700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 0x7f5a5003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:00.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.314+0000 7f5a69e00700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 0x7f5a5003b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006fd0 tx=0x7f5a58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:00.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.319+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a5c029360 con 0x7f5a6c109870 2024-03-11T19:12:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.543+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a6c10ddf0 con 0x7f5a50038db0 2024-03-11T19:12:00.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.545+0000 7f5a63400700 1 -- 172.21.15.72:0/46312941 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f5a6c10ddf0 con 0x7f5a50038db0 2024-03-11T19:12:00.563 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:12:00.563 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.547+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 msgr2=0x7f5a5003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.547+0000 7f5a71400700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 0x7f5a5003b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a58006fd0 tx=0x7f5a58006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.547+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 msgr2=0x7f5a6c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.547+0000 7f5a71400700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c004f40 tx=0x7f5a5c005020 comp rx=0 tx=0).stop 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 shutdown_connections 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5a50038db0 0x7f5a5003b270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 --2- 172.21.15.72:0/46312941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c109870 0x7f5a6c1991e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 >> 172.21.15.72:0/46312941 conn(0x7f5a6c0ff360 msgr2=0x7f5a6c10b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 shutdown_connections 2024-03-11T19:12:00.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:00.548+0000 7f5a71400700 1 -- 172.21.15.72:0/46312941 wait complete. 2024-03-11T19:12:01.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:01.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:00 smithi072 ceph-mon[31695]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:01.436 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi143 2024-03-11T19:12:01.436 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:12:01.436 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:12:01.454 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:12:01.454 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:01.511 INFO:tasks.cephadm:Adding host smithi143 to orchestrator... 2024-03-11T19:12:01.512 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch host add smithi143 2024-03-11T19:12:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:02.110 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:01 smithi072 ceph-mon[31695]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:12:03.132 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:02 smithi072 ceph-mon[31695]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:04.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.140+0000 7f9cab200700 1 -- 172.21.15.72:0/208978006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 msgr2=0x7f9cac10cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:04.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.140+0000 7f9cab200700 1 --2- 172.21.15.72:0/208978006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac10cd90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009910 tx=0x7f9c9c009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:04.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.141+0000 7f9cab200700 1 -- 172.21.15.72:0/208978006 shutdown_connections 2024-03-11T19:12:04.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.141+0000 7f9cab200700 1 --2- 172.21.15.72:0/208978006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac10cd90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:04.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.141+0000 7f9cab200700 1 -- 172.21.15.72:0/208978006 >> 172.21.15.72:0/208978006 conn(0x7f9cac077250 msgr2=0x7f9cac079670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:04.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.141+0000 7f9cab200700 1 -- 172.21.15.72:0/208978006 shutdown_connections 2024-03-11T19:12:04.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.141+0000 7f9cab200700 1 -- 172.21.15.72:0/208978006 wait complete. 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:04.146 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:04.147 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:03 smithi072 ceph-mon[31695]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.142+0000 7f9cab200700 1 Processor -- start 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.142+0000 7f9cab200700 1 -- start start 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.142+0000 7f9cab200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.142+0000 7f9cab200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac19d8f0 con 0x7f9cac10c9b0 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.142+0000 7f9ca9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.143+0000 7f9ca9e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33972/0 (socket says 172.21.15.72:33972) 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.143+0000 7f9ca9e00700 1 -- 172.21.15.72:0/1717343244 learned_addr learned my addr 172.21.15.72:0/1717343244 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.143+0000 7f9ca9e00700 1 -- 172.21.15.72:0/1717343244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0095f0 con 0x7f9cac10c9b0 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.143+0000 7f9ca9e00700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006010 tx=0x7f9c9c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.144+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c9c01d070 con 0x7f9cac10c9b0 2024-03-11T19:12:04.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.144+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c9c00bc30 con 0x7f9cac10c9b0 2024-03-11T19:12:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.144+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac19daf0 con 0x7f9cac10c9b0 2024-03-11T19:12:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.144+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c9c0039a0 con 0x7f9cac10c9b0 2024-03-11T19:12:04.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.144+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac19df10 con 0x7f9cac10c9b0 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.145+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0632a0 con 0x7f9cac10c9b0 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.145+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f9c9c004390 con 0x7f9cac10c9b0 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.146+0000 7f9ca2a00700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 0x7f9c9003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.146+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c9c04cb50 con 0x7f9cac10c9b0 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.146+0000 7f9ca9400700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 0x7f9c9003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:04.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.147+0000 7f9ca9400700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 0x7f9c9003b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:04.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.151+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c9c02ae20 con 0x7f9cac10c9b0 2024-03-11T19:12:04.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:04.386+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cac112040 con 0x7f9c90038db0 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:05.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:04 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:06.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:05 smithi072 ceph-mon[31695]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:07.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:06 smithi072 ceph-mon[31695]: Deploying cephadm binary to smithi143 2024-03-11T19:12:07.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:06 smithi072 ceph-mon[31695]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:09.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.177+0000 7f9ca2a00700 1 -- 172.21.15.72:0/1717343244 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9cac112040 con 0x7f9c90038db0 2024-03-11T19:12:09.181 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi143' with addr '172.21.15.143' 2024-03-11T19:12:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 msgr2=0x7f9c9003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 0x7f9c9003b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c98006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 msgr2=0x7f9cac19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006010 tx=0x7f9c9c004ff0 comp rx=0 tx=0).stop 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 shutdown_connections 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9c90038db0 0x7f9c9003b270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 --2- 172.21.15.72:0/1717343244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cac10c9b0 0x7f9cac19d3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.180+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 >> 172.21.15.72:0/1717343244 conn(0x7f9cac077250 msgr2=0x7f9cac10f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.181+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 shutdown_connections 2024-03-11T19:12:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:09.181+0000 7f9cab200700 1 -- 172.21.15.72:0/1717343244 wait complete. 2024-03-11T19:12:09.262 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:08 smithi072 ceph-mon[31695]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:09.907 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:12:10.354 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:10.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:10 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:10.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:10 smithi072 ceph-mon[31695]: Added host smithi143 2024-03-11T19:12:10.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:10 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:10.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:10 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:11.542 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:11 smithi072 ceph-mon[31695]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:11.542 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.485+0000 7fe472000700 1 -- 172.21.15.72:0/4051877136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 msgr2=0x7fe46c074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.485+0000 7fe472000700 1 --2- 172.21.15.72:0/4051877136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c074480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009910 tx=0x7fe45c009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.486+0000 7fe472000700 1 -- 172.21.15.72:0/4051877136 shutdown_connections 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.486+0000 7fe472000700 1 --2- 172.21.15.72:0/4051877136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c074480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.486+0000 7fe472000700 1 -- 172.21.15.72:0/4051877136 >> 172.21.15.72:0/4051877136 conn(0x7fe46c0fd150 msgr2=0x7fe46c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.486+0000 7fe472000700 1 -- 172.21.15.72:0/4051877136 shutdown_connections 2024-03-11T19:12:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.486+0000 7fe472000700 1 -- 172.21.15.72:0/4051877136 wait complete. 2024-03-11T19:12:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.487+0000 7fe472000700 1 Processor -- start 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.487+0000 7fe472000700 1 -- start start 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.487+0000 7fe472000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.487+0000 7fe472000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1996f0 con 0x7fe46c0740a0 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.488+0000 7fe46b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.488+0000 7fe46b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37706/0 (socket says 172.21.15.72:37706) 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.488+0000 7fe46b400700 1 -- 172.21.15.72:0/1235130588 learned_addr learned my addr 172.21.15.72:0/1235130588 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.488+0000 7fe46b400700 1 -- 172.21.15.72:0/1235130588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0095f0 con 0x7fe46c0740a0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.488+0000 7fe46b400700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.489+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe45c01d070 con 0x7fe46c0740a0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.489+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe45c00bc30 con 0x7fe46c0740a0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.489+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c1998f0 con 0x7fe46c0740a0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.489+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe45c0039a0 con 0x7fe46c0740a0 2024-03-11T19:12:12.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.489+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c199d10 con 0x7fe46c0740a0 2024-03-11T19:12:12.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.490+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fe45c004390 con 0x7fe46c0740a0 2024-03-11T19:12:12.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.490+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c0632a0 con 0x7fe46c0740a0 2024-03-11T19:12:12.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.490+0000 7fe463e00700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 0x7fe45003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:12.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.490+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe45c04cc00 con 0x7fe46c0740a0 2024-03-11T19:12:12.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.491+0000 7fe46aa00700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 0x7fe45003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:12.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.491+0000 7fe46aa00700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 0x7fe45003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:12.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.496+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe45c02ac00 con 0x7fe46c0740a0 2024-03-11T19:12:12.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.726+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe46c10ddd0 con 0x7fe450038d50 2024-03-11T19:12:12.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.728+0000 7fe463e00700 1 -- 172.21.15.72:0/1235130588 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fe46c10ddd0 con 0x7fe450038d50 2024-03-11T19:12:12.732 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:12:12.732 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}] 2024-03-11T19:12:12.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 msgr2=0x7fe45003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:12.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 0x7fe45003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:12.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 msgr2=0x7fe46c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:12.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c004ff0 comp rx=0 tx=0).stop 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 shutdown_connections 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe450038d50 0x7fe45003b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 --2- 172.21.15.72:0/1235130588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe46c0740a0 0x7fe46c1991b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 >> 172.21.15.72:0/1235130588 conn(0x7fe46c0fd150 msgr2=0x7fe46c1058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.731+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 shutdown_connections 2024-03-11T19:12:12.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:12.732+0000 7fe472000700 1 -- 172.21.15.72:0/1235130588 wait complete. 2024-03-11T19:12:13.091 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:12 smithi072 ceph-mon[31695]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:13.091 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:13.443 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-11T19:12:13.443 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-11T19:12:13.871 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:14.246 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:13 smithi072 ceph-mon[31695]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:12:15.078 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:14 smithi072 ceph-mon[31695]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:15.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.884+0000 7faa92e00700 1 -- 172.21.15.72:0/28305902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 msgr2=0x7faa8c109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.884+0000 7faa92e00700 1 --2- 172.21.15.72:0/28305902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c109a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa80009910 tx=0x7faa80009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.885+0000 7faa92e00700 1 -- 172.21.15.72:0/28305902 shutdown_connections 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.885+0000 7faa92e00700 1 --2- 172.21.15.72:0/28305902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c109a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.885+0000 7faa92e00700 1 -- 172.21.15.72:0/28305902 >> 172.21.15.72:0/28305902 conn(0x7faa8c0ff170 msgr2=0x7faa8c101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.885+0000 7faa92e00700 1 -- 172.21.15.72:0/28305902 shutdown_connections 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.885+0000 7faa92e00700 1 -- 172.21.15.72:0/28305902 wait complete. 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.886+0000 7faa92e00700 1 Processor -- start 2024-03-11T19:12:15.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.886+0000 7faa92e00700 1 -- start start 2024-03-11T19:12:15.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.886+0000 7faa92e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:15.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.886+0000 7faa92e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c199550 con 0x7faa8c109680 2024-03-11T19:12:15.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.887+0000 7faa91a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.887+0000 7faa91a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37718/0 (socket says 172.21.15.72:37718) 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.887+0000 7faa91a00700 1 -- 172.21.15.72:0/2178399199 learned_addr learned my addr 172.21.15.72:0/2178399199 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.887+0000 7faa91a00700 1 -- 172.21.15.72:0/2178399199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800095f0 con 0x7faa8c109680 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.887+0000 7faa91a00700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa80004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.888+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa8001d070 con 0x7faa8c109680 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.888+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa8000bbd0 con 0x7faa8c109680 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.888+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c1997b0 con 0x7faa8c109680 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.888+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa80003880 con 0x7faa8c109680 2024-03-11T19:12:15.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.888+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c199c30 con 0x7faa8c109680 2024-03-11T19:12:15.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.889+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c104dc0 con 0x7faa8c109680 2024-03-11T19:12:15.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.889+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7faa8000bd40 con 0x7faa8c109680 2024-03-11T19:12:15.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.890+0000 7faa8a000700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 0x7faa7403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:15.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.890+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa8004cb20 con 0x7faa8c109680 2024-03-11T19:12:15.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.890+0000 7faa91000700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 0x7faa7403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:15.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.891+0000 7faa91000700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 0x7faa7403b2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa78006fd0 tx=0x7faa78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:15.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:15.896+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa8002ae20 con 0x7faa8c109680 2024-03-11T19:12:16.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:16.129+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faa8c0632a0 con 0x7faa8c109680 2024-03-11T19:12:16.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:16.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:16 smithi072 ceph-mon[31695]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:12:17.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.079+0000 7faa8a000700 1 -- 172.21.15.72:0/2178399199 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7faa8002ac40 con 0x7faa8c109680 2024-03-11T19:12:17.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.083+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 msgr2=0x7faa7403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.083+0000 7faa92e00700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 0x7faa7403b2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faa78006fd0 tx=0x7faa78006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.083+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 msgr2=0x7faa8c199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.083+0000 7faa92e00700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa80004ff0 comp rx=0 tx=0).stop 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 shutdown_connections 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faa74038e00 0x7faa7403b2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 --2- 172.21.15.72:0/2178399199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faa8c109680 0x7faa8c199010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 >> 172.21.15.72:0/2178399199 conn(0x7faa8c0ff170 msgr2=0x7faa8c10b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 shutdown_connections 2024-03-11T19:12:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:17.084+0000 7faa92e00700 1 -- 172.21.15.72:0/2178399199 wait complete. 2024-03-11T19:12:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2178399199' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:17.822 INFO:tasks.cephadm:Adding mon.a on smithi072 2024-03-11T19:12:17.822 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-03-11T19:12:17.822 INFO:tasks.cephadm:Adding mon.c on smithi143 2024-03-11T19:12:17.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch apply mon '3;smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c' 2024-03-11T19:12:18.060 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:18.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:18 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2178399199' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-11T19:12:18.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:18 smithi072 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:19.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- 172.21.15.143:0/2720479259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 msgr2=0x7fcc68107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/2720479259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68107a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009910 tx=0x7fcc58009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- 172.21.15.143:0/2720479259 shutdown_connections 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/2720479259 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68107a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- 172.21.15.143:0/2720479259 >> 172.21.15.143:0/2720479259 conn(0x7fcc680fd180 msgr2=0x7fcc680ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- 172.21.15.143:0/2720479259 shutdown_connections 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- 172.21.15.143:0/2720479259 wait complete. 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 Processor -- start 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.358+0000 7fcc6fb76700 1 -- start start 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.359+0000 7fcc6fb76700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.359+0000 7fcc6fb76700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681974e0 con 0x7fcc68107650 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.359+0000 7fcc6d912700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.359+0000 7fcc6d912700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.143:33066/0 (socket says 172.21.15.143:33066) 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.359+0000 7fcc6d912700 1 -- 172.21.15.143:0/3672071855 learned_addr learned my addr 172.21.15.143:0/3672071855 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc6d912700 1 -- 172.21.15.143:0/3672071855 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580095f0 con 0x7fcc68107650 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc6d912700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006010 tx=0x7fcc58004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:19.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc5801d070 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68197740 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68197bc0 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc5800bbd0 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.360+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc58003810 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.361+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c005330 con 0x7fcc68107650 2024-03-11T19:12:19.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.362+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fcc5800bd40 con 0x7fcc68107650 2024-03-11T19:12:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.362+0000 7fcc5effd700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 0x7fcc5403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:19.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.363+0000 7fcc6d111700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 0x7fcc5403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:19.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.363+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc5800e840 con 0x7fcc68107650 2024-03-11T19:12:19.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.363+0000 7fcc6d111700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 0x7fcc5403b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006fd0 tx=0x7fcc64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:19.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.378+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc58003970 con 0x7fcc68107650 2024-03-11T19:12:19.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:19 smithi072 ceph-mon[31695]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.580+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc4c000c80 con 0x7fcc54038d50 2024-03-11T19:12:19.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.595+0000 7fcc5effd700 1 -- 172.21.15.143:0/3672071855 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcc4c000c80 con 0x7fcc54038d50 2024-03-11T19:12:19.599 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mon update... 2024-03-11T19:12:19.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.596+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 msgr2=0x7fcc5403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.596+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 0x7fcc5403b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006fd0 tx=0x7fcc64006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.596+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 msgr2=0x7fcc68196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.596+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006010 tx=0x7fcc58004ff0 comp rx=0 tx=0).stop 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 shutdown_connections 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc54038d50 0x7fcc5403b210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 --2- 172.21.15.143:0/3672071855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc68107650 0x7fcc68196fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 >> 172.21.15.143:0/3672071855 conn(0x7fcc680fd180 msgr2=0x7fcc68109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 shutdown_connections 2024-03-11T19:12:19.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:19.597+0000 7fcc6fb76700 1 -- 172.21.15.143:0/3672071855 wait complete. 2024-03-11T19:12:20.287 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.b.service 2024-03-11T19:12:20.289 DEBUG:teuthology.orchestra.run.smithi143:mon.c> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.c.service 2024-03-11T19:12:20.291 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:12:20.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c;count:3 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: Deploying daemon mon.c on smithi143 2024-03-11T19:12:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:20 smithi072 ceph-mon[31695]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:22 smithi072 ceph-mon[31695]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 -- 172.21.15.143:0/4058910826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 msgr2=0x7efd70107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 --2- 172.21.15.143:0/4058910826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70107ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009910 tx=0x7efd6c009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 -- 172.21.15.143:0/4058910826 shutdown_connections 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 --2- 172.21.15.143:0/4058910826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70107ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 -- 172.21.15.143:0/4058910826 >> 172.21.15.143:0/4058910826 conn(0x7efd700fb7c0 msgr2=0x7efd700fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 -- 172.21.15.143:0/4058910826 shutdown_connections 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.585+0000 7efd77514700 1 -- 172.21.15.143:0/4058910826 wait complete. 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd77514700 1 Processor -- start 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd77514700 1 -- start start 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd77514700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd77514700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70102b70 con 0x7efd701076d0 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd752b0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd752b0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.143:33080/0 (socket says 172.21.15.143:33080) 2024-03-11T19:12:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.586+0000 7efd752b0700 1 -- 172.21.15.143:0/753666469 learned_addr learned my addr 172.21.15.143:0/753666469 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:12:23.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.587+0000 7efd752b0700 1 -- 172.21.15.143:0/753666469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0095f0 con 0x7efd701076d0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.587+0000 7efd752b0700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd6c004f40 tx=0x7efd6c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.588+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd6c01d070 con 0x7efd701076d0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.588+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd701011c0 con 0x7efd701076d0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.588+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd70101640 con 0x7efd701076d0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.588+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd6c00bbd0 con 0x7efd701076d0 2024-03-11T19:12:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.588+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd6c003840 con 0x7efd701076d0 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.589+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd701021b0 con 0x7efd701076d0 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.589+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7efd6c00bd40 con 0x7efd701076d0 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.589+0000 7efd667fc700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 0x7efd5c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.589+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd6c04ca80 con 0x7efd701076d0 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.590+0000 7efd74aaf700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 0x7efd5c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:23.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.591+0000 7efd74aaf700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 0x7efd5c03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd60006fd0 tx=0x7efd60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:23.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.605+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd6c00b6f0 con 0x7efd701076d0 2024-03-11T19:12:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.900+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd70101940 con 0x7efd701076d0 2024-03-11T19:12:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.901+0000 7efd667fc700 1 -- 172.21.15.143:0/753666469 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7efd6c026020 con 0x7efd701076d0 2024-03-11T19:12:23.904 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:12:23.904 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":1,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","modified":"2024-03-11T19:10:19.252249Z","created":"2024-03-11T19:10:19.252249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 msgr2=0x7efd5c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 0x7efd5c03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd60006fd0 tx=0x7efd60006e40 comp rx=0 tx=0).stop 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 msgr2=0x7efd70100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd6c004f40 tx=0x7efd6c005020 comp rx=0 tx=0).stop 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 shutdown_connections 2024-03-11T19:12:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efd5c038d50 0x7efd5c03b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 --2- 172.21.15.143:0/753666469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd701076d0 0x7efd70100c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 >> 172.21.15.143:0/753666469 conn(0x7efd700fb7c0 msgr2=0x7efd700fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 shutdown_connections 2024-03-11T19:12:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:23.903+0000 7efd77514700 1 -- 172.21.15.143:0/753666469 wait complete. 2024-03-11T19:12:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 1 2024-03-11T19:12:24.331 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:23 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/753666469' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:12:24.890 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:24 smithi143 ceph-mon[36013]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-11T19:12:25.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:24 smithi143 bash[35979]: 406f788138dfdeb953b2d2242ebfcf3c266ab100960144558eafae772e967c0a 2024-03-11T19:12:25.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:24 smithi143 systemd[1]: Started Ceph mon.c for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:12:25.650 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:12:25.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:12:25.885 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:12:30.343 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:30.343 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: mon.a calling monitor election 2024-03-11T19:12:30.343 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.343 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: mon.c calling monitor election 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: monmap e2: 2 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: fsmap 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: mgrmap e13: x(active, since 59s) 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: overall HEALTH_OK 2024-03-11T19:12:30.344 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:30.345 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: mon.a calling monitor election 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: mon.c calling monitor election 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: monmap e2: 2 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: fsmap 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: mgrmap e13: x(active, since 59s) 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: overall HEALTH_OK 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:30.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 -- 172.21.15.143:0/2564124911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2680057d0 msgr2=0x7fd268007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 --2- 172.21.15.143:0/2564124911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2680057d0 0x7fd268007c60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd280069660 tx=0x7fd27000feb0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 -- 172.21.15.143:0/2564124911 shutdown_connections 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 --2- 172.21.15.143:0/2564124911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2680057d0 0x7fd268007c60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 -- 172.21.15.143:0/2564124911 >> 172.21.15.143:0/2564124911 conn(0x7fd280076d60 msgr2=0x7fd280077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 -- 172.21.15.143:0/2564124911 shutdown_connections 2024-03-11T19:12:30.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.448+0000 7fd28663d700 1 -- 172.21.15.143:0/2564124911 wait complete. 2024-03-11T19:12:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 Processor -- start 2024-03-11T19:12:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 -- start start 2024-03-11T19:12:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 0x7fd28019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280199dc0 con 0x7fd280199830 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd28663d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280199f30 con 0x7fd2680057d0 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49420/0 (socket says 172.21.15.143:49420) 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 -- 172.21.15.143:0/2175949103 learned_addr learned my addr 172.21.15.143:0/2175949103 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 -- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 msgr2=0x7fd2801992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 -- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 msgr2=0x7fd2801992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:12:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.449+0000 7fd27ffff700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.450+0000 7fd27f7fe700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 0x7fd28019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.450+0000 7fd27f7fe700 1 -- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 msgr2=0x7fd2801992f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.450+0000 7fd27f7fe700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.450+0000 7fd27f7fe700 1 -- 172.21.15.143:0/2175949103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27000f900 con 0x7fd280199830 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd27f7fe700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 0x7fd28019dbc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b4d0 tx=0x7fd27400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd274011630 con 0x7fd280199830 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28019e160 con 0x7fd280199830 2024-03-11T19:12:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd274011c70 con 0x7fd280199830 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd274010de0 con 0x7fd280199830 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.451+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28019e6b0 con 0x7fd280199830 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.452+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2801931e0 con 0x7fd280199830 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.453+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fd2740104a0 con 0x7fd280199830 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.453+0000 7fd27d7fa700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 0x7fd26c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.453+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd27404b940 con 0x7fd280199830 2024-03-11T19:12:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.453+0000 7fd27ffff700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 0x7fd26c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.454+0000 7fd27ffff700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 0x7fd26c03afb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd270005b40 tx=0x7fd270005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:30.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.469+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd274004550 con 0x7fd280199830 2024-03-11T19:12:30.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.740+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd28019a650 con 0x7fd280199830 2024-03-11T19:12:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.741+0000 7fd27d7fa700 1 -- 172.21.15.143:0/2175949103 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fd2740193b0 con 0x7fd280199830 2024-03-11T19:12:30.744 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:12:30.744 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","modified":"2024-03-11T19:12:24.856353Z","created":"2024-03-11T19:10:19.252249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:12:30.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 msgr2=0x7fd26c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:30.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 0x7fd26c03afb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd270005b40 tx=0x7fd270005a90 comp rx=0 tx=0).stop 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 msgr2=0x7fd28019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 0x7fd28019dbc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b4d0 tx=0x7fd27400b890 comp rx=0 tx=0).stop 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 shutdown_connections 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd26c038af0 0x7fd26c03afb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2680057d0 0x7fd2801992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 --2- 172.21.15.143:0/2175949103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd280199830 0x7fd28019dbc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 >> 172.21.15.143:0/2175949103 conn(0x7fd280076d60 msgr2=0x7fd2800fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 shutdown_connections 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:30.743+0000 7fd28663d700 1 -- 172.21.15.143:0/2175949103 wait complete. 2024-03-11T19:12:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-03-11T19:12:31.084 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:31.084 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:31.084 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:31.084 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:31.085 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: Deploying daemon mon.b on smithi106 2024-03-11T19:12:31.085 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/2175949103' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:12:31.085 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: Deploying daemon mon.b on smithi106 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/2175949103' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:12:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:32.398 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:12:32.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:12:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:12:33.020 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:32 smithi143 ceph-mon[36013]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:33.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:32 smithi072 ceph-mon[31695]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:33.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 podman[35871]: 2024-03-11 19:12:33.798521401 +0000 UTC m=+0.295171590 container init 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 podman[35871]: 2024-03-11 19:12:33.801776097 +0000 UTC m=+0.298426275 container start 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: set uid:gid to 167:167 (ceph:ceph) 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pidfile_write: ignore empty --pid-file 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: load: jerasure load: lrc 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: RocksDB version: 7.9.2 2024-03-11T19:12:34.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Git sha 0 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: DB SUMMARY 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: DB Session ID: RVFQMGN966QJW9U06CK0 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: CURRENT file: CURRENT 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: IDENTITY file: IDENTITY 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.error_if_exists: 0 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.create_if_missing: 0 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.paranoid_checks: 1 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-11T19:12:34.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.env: 0x55cc71c5cae0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.fs: PosixFileSystem 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.info_log: 0x55cc737818c0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_file_opening_threads: 16 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.statistics: (nil) 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.use_fsync: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_log_file_size: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.keep_log_file_num: 1000 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.recycle_log_file_num: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_fallocate: 1 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_mmap_reads: 0 2024-03-11T19:12:34.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_mmap_writes: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.use_direct_reads: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.create_missing_column_families: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.db_log_dir: 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.wal_dir: 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.advise_random_on_open: 1 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.db_write_buffer_size: 0 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.write_buffer_manager: 0x55cc73755400 2024-03-11T19:12:34.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.rate_limiter: (nil) 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.wal_recovery_mode: 2 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enable_thread_tracking: 0 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enable_pipelined_write: 0 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.unordered_write: 0 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.row_cache: None 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.wal_filter: None 2024-03-11T19:12:34.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_ingest_behind: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.two_write_queues: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.manual_wal_flush: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.wal_compression: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.atomic_flush: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.log_readahead_size: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.best_efforts_recovery: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.allow_data_in_errors: 0 2024-03-11T19:12:34.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.db_host_id: __hostname__ 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_background_jobs: 2 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_background_compactions: -1 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_subcompactions: 1 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_total_wal_size: 0 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_open_files: -1 2024-03-11T19:12:34.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bytes_per_sync: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_readahead_size: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_background_flushes: -1 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Compression algorithms supported: 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kZSTD supported: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kXpressCompression supported: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kLZ4HCCompression supported: 1 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kZlibCompression supported: 1 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kSnappyCompression supported: 1 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kLZ4Compression supported: 1 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: kBZip2Compression supported: 0 2024-03-11T19:12:34.154 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.merge_operator: 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_filter: None 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_filter_factory: None 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.sst_partitioner_factory: None 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cc737819c0) 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout: cache_index_and_filter_blocks: 1 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-11T19:12:34.155 INFO:journalctl@ceph.mon.b.smithi106.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: pin_top_level_index_and_filter: 1 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_type: 0 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: data_block_index_type: 0 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_shortening: 1 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: checksum: 4 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: no_block_cache: 0 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache: 0x55cc73785090 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_name: BinnedLRUCache 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_options: 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: capacity : 536870912 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: num_shard_bits : 4 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: strict_capacity_limit : 0 2024-03-11T19:12:34.156 INFO:journalctl@ceph.mon.b.smithi106.stdout: high_pri_pool_ratio: 0.000 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_cache_compressed: (nil) 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: persistent_cache: (nil) 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_size: 4096 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_size_deviation: 10 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_restart_interval: 16 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: index_block_restart_interval: 1 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: metadata_block_size: 4096 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: partition_filters: 0 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: use_delta_encoding: 1 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: filter_policy: bloomfilter 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: whole_key_filtering: 1 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: verify_compression: 0 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: read_amp_bytes_per_bit: 0 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: format_version: 5 2024-03-11T19:12:34.157 INFO:journalctl@ceph.mon.b.smithi106.stdout: enable_index_compression: 1 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout: block_align: 0 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout: max_auto_readahead_size: 262144 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout: prepopulate_block_cache: 0 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout: initial_auto_readahead_size: 8192 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout: num_file_reads_for_auto_readahead: 2 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.write_buffer_size: 33554432 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_write_buffer_number: 2 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression: NoCompression 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression: Disabled 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.prefix_extractor: nullptr 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.num_levels: 7 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-11T19:12:34.158 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.level: 32767 2024-03-11T19:12:34.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.strategy: 0 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.enabled: false 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.target_file_size_base: 67108864 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-11T19:12:34.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.arena_block_size: 1048576 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.disable_auto_compactions: 0 2024-03-11T19:12:34.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.table_properties_collectors: 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.inplace_update_support: 0 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-11T19:12:34.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.bloom_locality: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.max_successive_merges: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.paranoid_file_checks: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.force_consistency_checks: 1 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.report_bg_io_stats: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.ttl: 2592000 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enable_blob_files: false 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.min_blob_size: 0 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_file_size: 268435456 2024-03-11T19:12:34.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.blob_file_starting_level: 0 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 11a57cb3-d890-43b1-b4f6-e4a58cef5986 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184353830845, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184353831263, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710184353, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "11a57cb3-d890-43b1-b4f6-e4a58cef5986", "db_session_id": "RVFQMGN966QJW9U06CK0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-11T19:12:34.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710184353831352, "job": 1, "event": "recovery_finished"} 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cc73876000 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: DB pointer 0x55cc73860000 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** DB Stats ** 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:12:34.165 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** Compaction Stats [default] ** 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** Compaction Stats [default] ** 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-11T19:12:34.166 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Block cache BinnedLRUCache@0x55cc73785090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-11T19:12:34.167 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: using public_addr v2:172.21.15.106:0/0 -> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: starting mon.b rank -1 at public addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] at bind addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(???) e0 preinit fsid d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:12:34.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).mds e1 new map 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: e1 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: legacy client fscid: -1 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout: No filesystems configured 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-11T19:12:34.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Deploying cephadm binary to smithi106 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Added host smithi106 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.170 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Deploying cephadm binary to smithi143 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Added host smithi143 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2178399199' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2178399199' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c;count:3 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.174 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Deploying daemon mon.c on smithi143 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/753666469' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.a calling monitor election 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.c calling monitor election 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: monmap e2: 2 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: fsmap 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mgrmap e13: x(active, since 59s) 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: overall HEALTH_OK 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:34.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: Deploying daemon mon.b on smithi106 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/2175949103' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 ceph-mon[35889]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 bash[35871]: 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d 2024-03-11T19:12:34.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:33 smithi106 systemd[1]: Started Ceph mon.b for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:12:39.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: mon.c calling monitor election 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: mon.a calling monitor election 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: fsmap 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: mgrmap e13: x(active, since 68s) 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-11T19:12:39.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:39.192 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:39.192 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: mon.c calling monitor election 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: mon.a calling monitor election 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: fsmap 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: mgrmap e13: x(active, since 68s) 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:39.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 -- 172.21.15.143:0/2157650517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad00035c0 msgr2=0x7fbad0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 --2- 172.21.15.143:0/2157650517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad00035c0 0x7fbad0005a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbae4101680 tx=0x7fbae0009ee0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 -- 172.21.15.143:0/2157650517 shutdown_connections 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 --2- 172.21.15.143:0/2157650517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad00035c0 0x7fbad0005a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 -- 172.21.15.143:0/2157650517 >> 172.21.15.143:0/2157650517 conn(0x7fbae40fd240 msgr2=0x7fbae40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 -- 172.21.15.143:0/2157650517 shutdown_connections 2024-03-11T19:12:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.082+0000 7fbaec753700 1 -- 172.21.15.143:0/2157650517 wait complete. 2024-03-11T19:12:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.083+0000 7fbaec753700 1 Processor -- start 2024-03-11T19:12:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.083+0000 7fbaec753700 1 -- start start 2024-03-11T19:12:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 0x7fbae419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4197e20 con 0x7fbae419c000 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae41a3d70 con 0x7fbad00035c0 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae41a3eb0 con 0x7fbae4197730 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53226/0 (socket says 172.21.15.143:53226) 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 -- 172.21.15.143:0/3753769603 learned_addr learned my addr 172.21.15.143:0/3753769603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaea4ef700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaeacf0700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 0x7fbae419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaea4ef700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 msgr2=0x7fbae41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaea4ef700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 msgr2=0x7fbae41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaea4ef700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 msgr2=0x7fbae41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaea4ef700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 msgr2=0x7fbae419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 0x7fbae419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 -- 172.21.15.143:0/3753769603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0009930 con 0x7fbae4197730 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaeacf0700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 0x7fbae419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbae9cee700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b780 tx=0x7fbad400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004230 con 0x7fbae4197730 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41a4190 con 0x7fbae4197730 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.084+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbad4004390 con 0x7fbae4197730 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.085+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4011ba0 con 0x7fbae4197730 2024-03-11T19:12:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.085+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41a4690 con 0x7fbae4197730 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.086+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4190ef0 con 0x7fbae4197730 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.086+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fbad4003c10 con 0x7fbae4197730 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.086+0000 7fbadb7fe700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 0x7fbac803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.086+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbad404b410 con 0x7fbae4197730 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.087+0000 7fbaea4ef700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 0x7fbac803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.088+0000 7fbaea4ef700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 0x7fbac803b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbae00062a0 tx=0x7fbae0006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:40.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.091+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbad4004500 con 0x7fbae4197730 2024-03-11T19:12:40.188 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:40.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:40.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.369+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbae41985e0 con 0x7fbae4197730 2024-03-11T19:12:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.370+0000 7fbadb7fe700 1 -- 172.21.15.143:0/3753769603 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fbad4014160 con 0x7fbae4197730 2024-03-11T19:12:40.373 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-03-11T19:12:40.373 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","modified":"2024-03-11T19:12:33.854593Z","created":"2024-03-11T19:10:19.252249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 msgr2=0x7fbac803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 0x7fbac803b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbae00062a0 tx=0x7fbae0006230 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 msgr2=0x7fbae419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b780 tx=0x7fbad400bb40 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 shutdown_connections 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbac80391e0 0x7fbac803b6a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbae419c000 0x7fbae419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad00035c0 0x7fbae41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 --2- 172.21.15.143:0/3753769603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae4197730 0x7fbae419bac0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 >> 172.21.15.143:0/3753769603 conn(0x7fbae40fd240 msgr2=0x7fbae4106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 shutdown_connections 2024-03-11T19:12:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:40.372+0000 7fbaec753700 1 -- 172.21.15.143:0/3753769603 wait complete. 2024-03-11T19:12:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 3 2024-03-11T19:12:40.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:40.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:40.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.147 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-11T19:12:41.147 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: mon.c calling monitor election 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: mon.a calling monitor election 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: fsmap 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: mgrmap e13: x(active, since 68s) 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.149 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:41.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:41.563 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:41.860 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mon.b calling monitor election 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mon.b calling monitor election 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mon.c calling monitor election 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mon.a calling monitor election 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: fsmap 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:41.861 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: mgrmap e13: x(active, since 70s) 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: overall HEALTH_OK 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.862 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mon.b calling monitor election 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mon.b calling monitor election 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mon.c calling monitor election 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mon.a calling monitor election 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: fsmap 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: mgrmap e13: x(active, since 70s) 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: overall HEALTH_OK 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mon.b calling monitor election 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mon.b calling monitor election 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mon.c calling monitor election 2024-03-11T19:12:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mon.a calling monitor election 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: monmap e3: 3 mons at {a=[v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: fsmap 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: mgrmap e13: x(active, since 70s) 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: overall HEALTH_OK 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:41.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.982 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:42.983 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:42 smithi072 ceph-mon[31695]: Reconfiguring daemon mon.a on smithi072 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:12:43.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:43.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:43.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:43.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:42 smithi106 ceph-mon[35889]: Reconfiguring daemon mon.a on smithi072 2024-03-11T19:12:43.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:12:43.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:43.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:42 smithi143 ceph-mon[36013]: Reconfiguring daemon mon.a on smithi072 2024-03-11T19:12:44.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 -- 172.21.15.72:0/3496530140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c4107640 msgr2=0x7fc9c4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:44.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 --2- 172.21.15.72:0/3496530140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c4107640 0x7fc9c4107a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009910 tx=0x7fc9b4009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:44.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 -- 172.21.15.72:0/3496530140 shutdown_connections 2024-03-11T19:12:44.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 --2- 172.21.15.72:0/3496530140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c4107640 0x7fc9c4107a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 -- 172.21.15.72:0/3496530140 >> 172.21.15.72:0/3496530140 conn(0x7fc9c40fd150 msgr2=0x7fc9c40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:44.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 -- 172.21.15.72:0/3496530140 shutdown_connections 2024-03-11T19:12:44.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.683+0000 7fc9cb000700 1 -- 172.21.15.72:0/3496530140 wait complete. 2024-03-11T19:12:44.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.684+0000 7fc9cb000700 1 Processor -- start 2024-03-11T19:12:44.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 -- start start 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9c4107640 0x7fc9c4073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9c40744b0 0x7fc9c4074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a5d60 con 0x7fc9c41a3370 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a5ed0 con 0x7fc9c4107640 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.685+0000 7fc9cb000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c41a6090 con 0x7fc9c40744b0 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c3400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9c40744b0 0x7fc9c4074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9c4107640 0x7fc9c4073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:44.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49652/0 (socket says 172.21.15.72:49652) 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 -- 172.21.15.72:0/810781588 learned_addr learned my addr 172.21.15.72:0/810781588 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 -- 172.21.15.72:0/810781588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9c40744b0 msgr2=0x7fc9c4074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9c40744b0 0x7fc9c4074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 -- 172.21.15.72:0/810781588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9c4107640 msgr2=0x7fc9c4073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9c4107640 0x7fc9c4073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 -- 172.21.15.72:0/810781588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b40095f0 con 0x7fc9c41a3370 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.686+0000 7fc9c9000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00ba50 tx=0x7fc9bc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.687+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00c930 con 0x7fc9c41a3370 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.687+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc014440 con 0x7fc9c41a3370 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.687+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc007590 con 0x7fc9c41a3370 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.687+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c41a6370 con 0x7fc9c41a3370 2024-03-11T19:12:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.687+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c41a6870 con 0x7fc9c41a3370 2024-03-11T19:12:44.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.688+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fc9bc0145b0 con 0x7fc9c41a3370 2024-03-11T19:12:44.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.688+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c40632a0 con 0x7fc9c41a3370 2024-03-11T19:12:44.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.689+0000 7fc9c0c00700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 0x7fc9a803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:44.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.689+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc9bc04af90 con 0x7fc9c41a3370 2024-03-11T19:12:44.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.689+0000 7fc9c3e00700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 0x7fc9a803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:44.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.690+0000 7fc9c3e00700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 0x7fc9a803b6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4004690 tx=0x7fc9b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:44.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.694+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9bc012520 con 0x7fc9c41a3370 2024-03-11T19:12:44.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.913+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc9c410ba80 con 0x7fc9c41a3370 2024-03-11T19:12:44.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.914+0000 7fc9c0c00700 1 -- 172.21.15.72:0/810781588 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fc9bc012340 con 0x7fc9c41a3370 2024-03-11T19:12:44.918 INFO:teuthology.orchestra.run.smithi072.stdout:# minimal ceph.conf for d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:12:44.918 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2024-03-11T19:12:44.918 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:12:44.918 INFO:teuthology.orchestra.run.smithi072.stdout: mon_host = [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-03-11T19:12:44.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.916+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 msgr2=0x7fc9a803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:44.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.916+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 0x7fc9a803b6a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4004690 tx=0x7fc9b4005fb0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.916+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 msgr2=0x7fc9c41a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:44.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.916+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00ba50 tx=0x7fc9bc00bd60 comp rx=0 tx=0).stop 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 shutdown_connections 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc9a80391e0 0x7fc9a803b6a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9c41a3370 0x7fc9c41a5760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9c4107640 0x7fc9c4073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 --2- 172.21.15.72:0/810781588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9c40744b0 0x7fc9c4074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 >> 172.21.15.72:0/810781588 conn(0x7fc9c40fd150 msgr2=0x7fc9c4103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 shutdown_connections 2024-03-11T19:12:44.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:44.917+0000 7fc9cb000700 1 -- 172.21.15.72:0/810781588 wait complete. 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:44 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/810781588' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:44 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/810781588' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:44 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/810781588' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:45.606 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-11T19:12:45.606 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:12:45.607 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:12:45.697 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:12:45.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:45.822 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:12:45.822 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:12:45.853 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:12:45.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:45.921 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:12:45.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:12:45.957 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:12:45.957 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:12:46.027 INFO:tasks.cephadm:Adding mgr.x on smithi072 2024-03-11T19:12:46.027 INFO:tasks.cephadm:Adding mgr.y on smithi106 2024-03-11T19:12:46.028 INFO:tasks.cephadm:Adding mgr.z on smithi143 2024-03-11T19:12:46.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch apply mgr '3;smithi072=x;smithi106=y;smithi143=z' 2024-03-11T19:12:46.067 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:46 smithi072 ceph-mon[31695]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:12:46.067 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:46 smithi072 ceph-mon[31695]: Reconfiguring daemon mon.b on smithi106 2024-03-11T19:12:46.144 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:46 smithi106 ceph-mon[35889]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:12:46.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:46 smithi106 ceph-mon[35889]: Reconfiguring daemon mon.b on smithi106 2024-03-11T19:12:46.309 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:12:46.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:46 smithi143 ceph-mon[36013]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:12:46.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:46 smithi143 ceph-mon[36013]: Reconfiguring daemon mon.b on smithi106 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:47 smithi106 ceph-mon[35889]: Reconfiguring daemon mon.c on smithi143 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:47 smithi072 ceph-mon[31695]: Reconfiguring daemon mon.c on smithi143 2024-03-11T19:12:47.430 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:47.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:47 smithi143 ceph-mon[36013]: Reconfiguring daemon mon.c on smithi143 2024-03-11T19:12:48.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 -- 172.21.15.143:0/2286109515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91201006d0 msgr2=0x7f9120100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 --2- 172.21.15.143:0/2286109515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91201006d0 0x7f9120100af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9110009910 tx=0x7f9110009c20 comp rx=0 tx=0).stop 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 -- 172.21.15.143:0/2286109515 shutdown_connections 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 --2- 172.21.15.143:0/2286109515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91201006d0 0x7f9120100af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 -- 172.21.15.143:0/2286109515 >> 172.21.15.143:0/2286109515 conn(0x7f91200fbc70 msgr2=0x7f91200fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 -- 172.21.15.143:0/2286109515 shutdown_connections 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.538+0000 7f912618c700 1 -- 172.21.15.143:0/2286109515 wait complete. 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 Processor -- start 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 -- start start 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91200fc260 0x7f91200fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91200fcc20 0x7f91201a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a1380 con 0x7f91200fcc20 2024-03-11T19:12:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a14c0 con 0x7f91200fc260 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f912618c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a1600 con 0x7f91201006d0 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f911f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49432/0 (socket says 172.21.15.143:49432) 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.539+0000 7f911f7fe700 1 -- 172.21.15.143:0/2980471472 learned_addr learned my addr 172.21.15.143:0/2980471472 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 -- 172.21.15.143:0/2980471472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91200fc260 msgr2=0x7f91200fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91200fc260 0x7f91200fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 -- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91200fcc20 msgr2=0x7f91201a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91200fcc20 0x7f91201a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 -- 172.21.15.143:0/2980471472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100095f0 con 0x7f91201006d0 2024-03-11T19:12:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911f7fe700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f911000bcf0 tx=0x7f911000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911001d070 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9110004f40 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91201a1880 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000f460 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.540+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a1da0 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.541+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f911000f5c0 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.541+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91200632a0 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.541+0000 7f911cff9700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 0x7f910803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.541+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f911004cfe0 con 0x7f91201006d0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.542+0000 7f911effd700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 0x7f910803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.543+0000 7f911effd700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 0x7f910803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91140098e0 tx=0x7f9114006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.546+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9110020c60 con 0x7f91201006d0 2024-03-11T19:12:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.756+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi106=y;smithi143=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f912002d950 con 0x7f91080391e0 2024-03-11T19:12:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.769+0000 7f911cff9700 1 -- 172.21.15.143:0/2980471472 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f912002d950 con 0x7f91080391e0 2024-03-11T19:12:48.773 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mgr update... 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 msgr2=0x7f910803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 0x7f910803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91140098e0 tx=0x7f9114006dd0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 msgr2=0x7f91200fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f911000bcf0 tx=0x7f911000bdd0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 shutdown_connections 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91080391e0 0x7f910803b6a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91200fcc20 0x7f91201a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91201006d0 0x7f91200fdc10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 --2- 172.21.15.143:0/2980471472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91200fc260 0x7f91200fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 >> 172.21.15.143:0/2980471472 conn(0x7f91200fbc70 msgr2=0x7f912006a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 shutdown_connections 2024-03-11T19:12:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:12:48.771+0000 7f912618c700 1 -- 172.21.15.143:0/2980471472 wait complete. 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:49.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:49.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:49.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:49.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:49.497 DEBUG:teuthology.orchestra.run.smithi106:mgr.y> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.y.service 2024-03-11T19:12:49.499 DEBUG:teuthology.orchestra.run.smithi143:mgr.z> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z.service 2024-03-11T19:12:49.501 INFO:tasks.cephadm:Deploying OSDs... 2024-03-11T19:12:49.501 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:12:49.502 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:12:49.556 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:12:49.556 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-03-11T19:12:49.604 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:12:49.604 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:49.604 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 630 Links: 1 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-03-11 19:11:45.936059079 +0000 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-03-11 19:06:10.579337172 +0000 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-03-11 19:06:10.579337172 +0000 2024-03-11T19:12:49.605 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-03-11 19:06:10.579337172 +0000 2024-03-11T19:12:49.605 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:12:49.695 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-03-11T19:12:49.695 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-03-11T19:12:49.695 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000366134 s, 1.4 MB/s 2024-03-11T19:12:49.699 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:12:49.753 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:49.803 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-03-11 19:11:45.982058103 +0000 2024-03-11T19:12:49.804 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-03-11 19:06:11.164324552 +0000 2024-03-11T19:12:49.804 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-03-11 19:06:11.164324552 +0000 2024-03-11T19:12:49.804 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-03-11 19:06:11.164324552 +0000 2024-03-11T19:12:49.804 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:12:49.891 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-03-11T19:12:49.891 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-03-11T19:12:49.891 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000353215 s, 1.4 MB/s 2024-03-11T19:12:49.895 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:12:49.947 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-03-11T19:12:49.994 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:12:49.994 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:49.994 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-03-11 19:11:46.034056999 +0000 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-03-11 19:06:11.722312515 +0000 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-03-11 19:06:11.722312515 +0000 2024-03-11T19:12:49.995 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-03-11 19:06:11.722312515 +0000 2024-03-11T19:12:49.995 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:12:50.128 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-03-11T19:12:50.128 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-03-11T19:12:50.128 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000326374 s, 1.6 MB/s 2024-03-11T19:12:50.132 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:12:50.184 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-03-11T19:12:50.230 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:12:50.230 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-03-11 19:11:46.083055958 +0000 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-03-11 19:06:12.284300392 +0000 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-03-11 19:06:12.284300392 +0000 2024-03-11T19:12:50.231 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-03-11 19:06:12.284300392 +0000 2024-03-11T19:12:50.231 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:12:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-03-11T19:12:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-03-11T19:12:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000326556 s, 1.6 MB/s 2024-03-11T19:12:50.327 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:12:50.383 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-11T19:12:50.383 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:12:50.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:50 smithi106 ceph-mon[35889]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi106=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:50.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:50 smithi106 ceph-mon[35889]: Saving service mgr spec with placement smithi072=x;smithi106=y;smithi143=z;count:3 2024-03-11T19:12:50.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:50 smithi106 ceph-mon[35889]: Deploying daemon mgr.z on smithi143 2024-03-11T19:12:50.402 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:12:50.402 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-03-11T19:12:50.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:50 smithi072 ceph-mon[31695]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi106=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:50.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:50 smithi072 ceph-mon[31695]: Saving service mgr spec with placement smithi072=x;smithi106=y;smithi143=z;count:3 2024-03-11T19:12:50.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:50 smithi072 ceph-mon[31695]: Deploying daemon mgr.z on smithi143 2024-03-11T19:12:50.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:50 smithi143 ceph-mon[36013]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi106=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:12:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:50 smithi143 ceph-mon[36013]: Saving service mgr spec with placement smithi072=x;smithi106=y;smithi143=z;count:3 2024-03-11T19:12:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:50 smithi143 ceph-mon[36013]: Deploying daemon mgr.z on smithi143 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:50.459 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-11 19:12:33.315506403 +0000 2024-03-11T19:12:50.460 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-11 19:04:21.108523519 +0000 2024-03-11T19:12:50.460 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-11 19:04:21.108523519 +0000 2024-03-11T19:12:50.460 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-11 19:04:21.108523519 +0000 2024-03-11T19:12:50.460 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:12:50.526 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-11T19:12:50.527 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-11T19:12:50.527 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000274992 s, 1.9 MB/s 2024-03-11T19:12:50.528 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:12:50.584 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-03-11T19:12:50.639 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:12:50.639 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:50.639 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-11T19:12:50.639 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:50.639 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:50.640 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-11 19:12:33.315506403 +0000 2024-03-11T19:12:50.640 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-11 19:04:21.442515034 +0000 2024-03-11T19:12:50.640 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-11 19:04:21.442515034 +0000 2024-03-11T19:12:50.640 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-11 19:04:21.442515034 +0000 2024-03-11T19:12:50.640 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:12:50.705 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-11T19:12:50.706 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-11T19:12:50.706 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000209818 s, 2.4 MB/s 2024-03-11T19:12:50.707 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:12:50.765 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-11 19:12:33.316506378 +0000 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-11 19:04:21.785506322 +0000 2024-03-11T19:12:50.819 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-11 19:04:21.785506322 +0000 2024-03-11T19:12:50.820 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-11 19:04:21.785506322 +0000 2024-03-11T19:12:50.820 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:12:50.885 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-11T19:12:50.885 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-11T19:12:50.885 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000217192 s, 2.4 MB/s 2024-03-11T19:12:50.886 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:12:50.941 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-03-11T19:12:50.997 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:12:50.997 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:50.997 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-11T19:12:50.997 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:50.997 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:50.998 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-11 19:12:33.316506378 +0000 2024-03-11T19:12:50.998 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-11 19:04:22.136497407 +0000 2024-03-11T19:12:50.998 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-11 19:04:22.136497407 +0000 2024-03-11T19:12:50.998 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-11 19:04:22.136497407 +0000 2024-03-11T19:12:50.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:12:51.063 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-11T19:12:51.064 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-11T19:12:51.064 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000210765 s, 2.4 MB/s 2024-03-11T19:12:51.065 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:12:51.118 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:50 smithi143 systemd[1]: Starting Ceph mgr.z for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:12:51.121 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-03-11T19:12:51.121 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:12:51.136 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:12:51.136 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-03-11T19:12:51.190 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:12:51.190 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-03-11 19:12:50.842748137 +0000 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-03-11 19:04:23.933486455 +0000 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-03-11 19:04:23.933486455 +0000 2024-03-11T19:12:51.191 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-03-11 19:04:23.933486455 +0000 2024-03-11T19:12:51.191 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:12:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-03-11T19:12:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-03-11T19:12:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000250956 s, 2.0 MB/s 2024-03-11T19:12:51.276 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:12:51.291 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-03-11 19:12:50.843748109 +0000 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-03-11 19:04:24.284476417 +0000 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-03-11 19:04:24.284476417 +0000 2024-03-11T19:12:51.347 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-03-11 19:04:24.284476417 +0000 2024-03-11T19:12:51.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:12:51.387 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:51 smithi143 ceph-mon[36013]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:51.387 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 podman[37631]: 2024-03-11 19:12:51.022027998 +0000 UTC m=+0.017757426 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:12:51.387 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 podman[37631]: 2024-03-11 19:12:51.123409139 +0000 UTC m=+0.119138561 container create 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-11T19:12:51.387 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 podman[37631]: 2024-03-11 19:12:51.384520454 +0000 UTC m=+0.380249893 container init 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:12:51.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:51 smithi106 ceph-mon[35889]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:51.412 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-03-11T19:12:51.412 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-03-11T19:12:51.412 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000221508 s, 2.3 MB/s 2024-03-11T19:12:51.413 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:12:51.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:51 smithi072 ceph-mon[31695]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:51.469 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-03-11 19:12:50.843748109 +0000 2024-03-11T19:12:51.523 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-03-11 19:04:24.630466521 +0000 2024-03-11T19:12:51.524 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-03-11 19:04:24.630466521 +0000 2024-03-11T19:12:51.524 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-03-11 19:04:24.630466521 +0000 2024-03-11T19:12:51.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:12:51.589 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-03-11T19:12:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-03-11T19:12:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.00023953 s, 2.1 MB/s 2024-03-11T19:12:51.591 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:12:51.647 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-03-11T19:12:51.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 podman[37631]: 2024-03-11 19:12:51.387870313 +0000 UTC m=+0.383599747 container start 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-11T19:12:51.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 ceph-mgr[37738]: -- 172.21.15.143:0/4008368227 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b811eb61a0 con 0x55b811e8cc00 2024-03-11T19:12:51.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 bash[37631]: 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d 2024-03-11T19:12:51.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 systemd[1]: Started Ceph mgr.z for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:12:51.702 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-03-11 19:12:50.843748109 +0000 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-03-11 19:04:24.972456740 +0000 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-03-11 19:04:24.972456740 +0000 2024-03-11T19:12:51.703 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-03-11 19:04:24.972456740 +0000 2024-03-11T19:12:51.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:12:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-03-11T19:12:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-03-11T19:12:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000236805 s, 2.2 MB/s 2024-03-11T19:12:51.769 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:12:51.825 INFO:tasks.cephadm:Deploying osd.0 on smithi072 with /dev/vg_nvme/lv_4... 2024-03-11T19:12:51.825 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:12:52.036 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:51.941+0000 7f573dec2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:12:52.036 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:52.032+0000 7f573dec2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:12:52.255 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:52.676 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.676 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.676 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: Deploying daemon mgr.y on smithi106 2024-03-11T19:12:52.677 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:52 smithi072 ceph-mon[31695]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:52.835 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:52.664+0000 7f573dec2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:12:52.835 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:52.754+0000 7f573dec2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: Deploying daemon mgr.y on smithi106 2024-03-11T19:12:52.836 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-mon[36013]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:52.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: Deploying daemon mgr.y on smithi106 2024-03-11T19:12:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:52 smithi106 ceph-mon[35889]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:53.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:52.896+0000 7f573dec2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:12:53.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:52.990+0000 7f573dec2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:12:53.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.071+0000 7f573dec2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:12:53.621 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.332+0000 7f573dec2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:12:53.621 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.440+0000 7f573dec2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:12:53.621 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.524+0000 7f573dec2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:12:53.621 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.616+0000 7f573dec2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:12:53.795 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:53 smithi106 systemd[1]: Starting Ceph mgr.y for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:12:53.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:53.791+0000 7f573dec2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:12:54.060 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:53 smithi106 podman[37263]: 2024-03-11 19:12:53.694690198 +0000 UTC m=+0.017399896 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:12:54.060 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:53 smithi106 podman[37263]: 2024-03-11 19:12:53.811755555 +0000 UTC m=+0.134465259 container create 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-11T19:12:54.060 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 podman[37263]: 2024-03-11 19:12:54.030646198 +0000 UTC m=+0.353355874 container init 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T19:12:54.060 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 podman[37263]: 2024-03-11 19:12:54.033474543 +0000 UTC m=+0.356184228 container start 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T19:12:54.394 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 ceph-mgr[37282]: -- 172.21.15.106:0/2914436027 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a339221a0 con 0x564a338f9000 2024-03-11T19:12:54.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 bash[37263]: 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 2024-03-11T19:12:54.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 systemd[1]: Started Ceph mgr.y for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:12:54.894 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:54.580+0000 7f2736b9e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:12:54.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:54.678+0000 7f2736b9e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0352123 s, 298 MB/s 2024-03-11T19:12:55.010 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-03-11T19:12:55.188 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:54.782+0000 7f573dec2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:12:55.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:54.916+0000 7f573dec2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:55.275+0000 7f573dec2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:55.368+0000 7f573dec2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:55.349+0000 7f2736b9e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:55.447+0000 7f2736b9e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:12:55.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:55.761 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:55.568+0000 7f573dec2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:12:55.761 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:55.756+0000 7f573dec2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:12:55.780 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:55.589+0000 7f2736b9e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:12:55.780 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:55.691+0000 7f2736b9e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:12:55.780 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:55.776+0000 7f2736b9e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:12:56.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-03-11T19:12:56.312 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:56.044+0000 7f573dec2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:12:56.313 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:56.308+0000 7f573dec2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:12:56.342 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:56.048+0000 7f2736b9e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:12:56.342 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:56.160+0000 7f2736b9e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:12:56.343 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:56.245+0000 7f2736b9e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:12:56.343 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:56.339+0000 7f2736b9e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:12:56.622 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:12:56.644 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:56.511+0000 7f2736b9e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:12:56.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:56.541+0000 7f573dec2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:57.000 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.001 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:56 smithi072 ceph-mon[31695]: Reconfiguring daemon mgr.x on smithi072 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:56 smithi106 ceph-mon[35889]: Reconfiguring daemon mgr.x on smithi072 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:12:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:12:57.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:12:57.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:56 smithi143 ceph-mon[36013]: Reconfiguring daemon mgr.x on smithi072 2024-03-11T19:12:57.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:57.330+0000 7f573dec2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:12:57.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:57.473+0000 7f2736b9e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:12:57.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:57.602+0000 7f2736b9e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:12:58.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:57.896+0000 7f573dec2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:12:58.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:57.993+0000 7f573dec2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:12:58.231 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:57.943+0000 7f2736b9e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:12:58.232 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:58.032+0000 7f2736b9e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:12:58.232 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:58.227+0000 7f2736b9e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:12:58.644 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:58.409+0000 7f2736b9e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:12:58.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:12:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:12:58.204+0000 7f573dec2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:12:58.947 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:58.690+0000 7f2736b9e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:12:58.947 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:58.943+0000 7f2736b9e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: Standby manager daemon z started 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:12:59.157 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:12:59 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:12:59.394 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:59.172+0000 7f2736b9e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: Standby manager daemon z started 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:12:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: Standby manager daemon z started 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:12:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:12:59 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2849733443' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:12:59.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 -- 172.21.15.72:0/3771030584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98102740 msgr2=0x7f1c98108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:59.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 --2- 172.21.15.72:0/3771030584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98102740 0x7f1c98108ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b0d0 tx=0x7f1c9000b3e0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 -- 172.21.15.72:0/3771030584 shutdown_connections 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 --2- 172.21.15.72:0/3771030584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98102740 0x7f1c98108ff0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 --2- 172.21.15.72:0/3771030584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c98100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 --2- 172.21.15.72:0/3771030584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98101da0 0x7f1c98102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 -- 172.21.15.72:0/3771030584 >> 172.21.15.72:0/3771030584 conn(0x7f1c980fc140 msgr2=0x7f1c980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 -- 172.21.15.72:0/3771030584 shutdown_connections 2024-03-11T19:12:59.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.644+0000 7f1c9d000700 1 -- 172.21.15.72:0/3771030584 wait complete. 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.645+0000 7f1c9d000700 1 Processor -- start 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.645+0000 7f1c9d000700 1 -- start start 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 0x7f1c9819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98102740 0x7f1c981a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981a1b90 con 0x7f1c98101da0 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981a1cd0 con 0x7f1c98100ba0 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c9d000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981a1e60 con 0x7f1c98102740 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:59.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52182/0 (socket says 172.21.15.72:52182) 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 -- 172.21.15.72:0/2354854298 learned_addr learned my addr 172.21.15.72:0/2354854298 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95400700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 0x7f1c9819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c96800700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98102740 0x7f1c981a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 -- 172.21.15.72:0/2354854298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98102740 msgr2=0x7f1c981a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98102740 0x7f1c981a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 -- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 msgr2=0x7f1c9819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 0x7f1c9819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95e00700 1 -- 172.21.15.72:0/2354854298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c88009520 con 0x7f1c98100ba0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.646+0000 7f1c95400700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 0x7f1c9819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c95e00700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800ea50 tx=0x7f1c8800ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800cc90 con 0x7f1c98100ba0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c8800cdf0 con 0x7f1c98100ba0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c90009d00 con 0x7f1c98100ba0 2024-03-11T19:12:59.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88018840 con 0x7f1c98100ba0 2024-03-11T19:12:59.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.647+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c981a2500 con 0x7f1c98100ba0 2024-03-11T19:12:59.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.648+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c98068930 con 0x7f1c98100ba0 2024-03-11T19:12:59.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.649+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 14) v1 ==== 91024+0+0 (secure 0 0 0) 0x7f1c88010430 con 0x7f1c98100ba0 2024-03-11T19:12:59.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.650+0000 7f1c8ea00700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 0x7f1c7c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:12:59.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.650+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c88014070 con 0x7f1c98100ba0 2024-03-11T19:12:59.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.650+0000 7f1c95400700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 0x7f1c7c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:12:59.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.651+0000 7f1c95400700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 0x7f1c7c06f9a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009cb0 tx=0x7f1c84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:12:59.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.654+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c88056d50 con 0x7f1c98100ba0 2024-03-11T19:12:59.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:12:59.883+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c980fe2c0 con 0x7f1c7c06d4e0 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:00 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:00.394 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:12:59 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:12:59.982+0000 7f2736b9e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:00 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:00.865 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:13:00.551+0000 7f2736b9e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:13:00.865 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:13:00.647+0000 7f2736b9e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:13:00.865 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:13:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:13:00.861+0000 7f2736b9e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: Standby manager daemon y started 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:13:01.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:13:01.106 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:01 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:13:01.144 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: Standby manager daemon y started 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:13:01.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:01 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:13:01.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:01.371+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f1c88057120 con 0x7f1c98100ba0 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: Standby manager daemon y started 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:13:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:01 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1855571320' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:13:02.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:02 smithi072 ceph-mon[31695]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-11T19:13:02.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:02 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:13:02.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:02 smithi072 ceph-mon[31695]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:02.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:02 smithi106 ceph-mon[35889]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-11T19:13:02.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:02 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:13:02.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:02 smithi106 ceph-mon[35889]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:02 smithi143 ceph-mon[36013]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-11T19:13:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:02 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:13:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:02 smithi143 ceph-mon[36013]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:04.240 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:03 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2313566876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.240 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:03 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.241 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:03 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]': finished 2024-03-11T19:13:04.241 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:03 smithi072 ceph-mon[31695]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:13:04.241 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:03 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2313566876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:03 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:03 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]': finished 2024-03-11T19:13:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:03 smithi106 ceph-mon[35889]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:13:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:03 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2313566876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:03 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]: dispatch 2024-03-11T19:13:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:03 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cbac0ed-5153-4a8d-b65d-ac1a30323f24"}]': finished 2024-03-11T19:13:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:03 smithi143 ceph-mon[36013]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:13:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:04 smithi106 ceph-mon[35889]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1030619928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:04 smithi072 ceph-mon[31695]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1030619928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:04 smithi143 ceph-mon[36013]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1030619928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:06 smithi106 ceph-mon[35889]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:06 smithi072 ceph-mon[31695]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:06 smithi143 ceph-mon[36013]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:09.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:08 smithi072 ceph-mon[31695]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:09.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:08 smithi106 ceph-mon[35889]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:08 smithi143 ceph-mon[36013]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:11.261 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:10 smithi072 ceph-mon[31695]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:11.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:10 smithi106 ceph-mon[35889]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:10 smithi143 ceph-mon[36013]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:13.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:12 smithi072 ceph-mon[31695]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:13.241 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:12 smithi143 ceph-mon[36013]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:13.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:12 smithi106 ceph-mon[35889]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:14.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:13 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:13:14.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:13 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:14.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:13 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:13:14.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:13 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:13 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:13:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:13 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:15.262 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:14 smithi072 ceph-mon[31695]: Deploying daemon osd.0 on smithi072 2024-03-11T19:13:15.262 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:14 smithi072 ceph-mon[31695]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:14 smithi106 ceph-mon[35889]: Deploying daemon osd.0 on smithi072 2024-03-11T19:13:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:14 smithi106 ceph-mon[35889]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:15.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:14 smithi143 ceph-mon[36013]: Deploying daemon osd.0 on smithi072 2024-03-11T19:13:15.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:14 smithi143 ceph-mon[36013]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:17.325 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:16 smithi072 ceph-mon[31695]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:17.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:16 smithi106 ceph-mon[35889]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:16 smithi143 ceph-mon[36013]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:19.094 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:18 smithi072 ceph-mon[31695]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:18 smithi106 ceph-mon[35889]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:18 smithi143 ceph-mon[36013]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:21.303 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:20 smithi072 ceph-mon[31695]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:21.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:21 smithi106 ceph-mon[35889]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:21 smithi143 ceph-mon[36013]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:23.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:22 smithi072 ceph-mon[31695]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:23.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:22 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:22 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:22 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:22 smithi106 ceph-mon[35889]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:22 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:22 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:22 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:22 smithi143 ceph-mon[36013]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:22 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:22 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:22 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:25.259 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:24 smithi072 ceph-mon[31695]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:25.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:24 smithi106 ceph-mon[35889]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:24 smithi143 ceph-mon[36013]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:13:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:26 smithi106 ceph-mon[35889]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:13:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:26 smithi072 ceph-mon[31695]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:13:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:26 smithi143 ceph-mon[36013]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:27.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.130+0000 7f1c8ea00700 1 -- 172.21.15.72:0/2354854298 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1c980fe2c0 con 0x7f1c7c06d4e0 2024-03-11T19:13:27.135 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 0 on host 'smithi072' 2024-03-11T19:13:27.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 msgr2=0x7f1c7c06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:27.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 0x7f1c7c06f9a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009cb0 tx=0x7f1c84009260 comp rx=0 tx=0).stop 2024-03-11T19:13:27.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 msgr2=0x7f1c9819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:27.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800ea50 tx=0x7f1c8800ee10 comp rx=0 tx=0).stop 2024-03-11T19:13:27.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 shutdown_connections 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1c7c06d4e0 0x7f1c7c06f9a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c98102740 0x7f1c981a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c98100ba0 0x7f1c9819a000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 --2- 172.21.15.72:0/2354854298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c98101da0 0x7f1c9819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 >> 172.21.15.72:0/2354854298 conn(0x7f1c980fc140 msgr2=0x7f1c980fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 shutdown_connections 2024-03-11T19:13:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:13:27.134+0000 7f1c9d000700 1 -- 172.21.15.72:0/2354854298 wait complete. 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:27 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.897 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:27 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:27 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:27.943 DEBUG:teuthology.orchestra.run.smithi072:osd.0> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.0.service 2024-03-11T19:13:27.947 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/vg_nvme/lv_4... 2024-03-11T19:13:27.947 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:13:28.189 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:13:28.637 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:28 smithi106 ceph-mon[35889]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:13:28.637 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:28 smithi106 ceph-mon[35889]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:13:28.637 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:28 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.637 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:28 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.637 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:28 smithi106 ceph-mon[35889]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:28 smithi072 ceph-mon[31695]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:13:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:28 smithi072 ceph-mon[31695]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:13:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:28 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:28 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:28 smithi072 ceph-mon[31695]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:28 smithi143 ceph-mon[36013]: from='osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:13:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:28 smithi143 ceph-mon[36013]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:13:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:28 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:28 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:28 smithi143 ceph-mon[36013]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:13:29.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:29 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:13:29.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:29 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:13:29.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:29 smithi106 ceph-mon[35889]: osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] boot 2024-03-11T19:13:29.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:29 smithi106 ceph-mon[35889]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:13:29.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:29 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:29.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:29 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:13:29.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:29 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:13:29.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:29 smithi072 ceph-mon[31695]: osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] boot 2024-03-11T19:13:29.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:29 smithi072 ceph-mon[31695]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:13:29.904 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:29 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:29 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:13:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:29 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:13:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:29 smithi143 ceph-mon[36013]: osd.0 [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] boot 2024-03-11T19:13:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:29 smithi143 ceph-mon[36013]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:13:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:29 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129411 s, 810 MB/s 2024-03-11T19:13:30.090 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-11T19:13:30.704 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:30 smithi106 ceph-mon[35889]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:13:30.704 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:30 smithi106 ceph-mon[35889]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:30.705 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-03-11T19:13:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:30 smithi072 ceph-mon[31695]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:13:30.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:30 smithi072 ceph-mon[31695]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:30.939 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:13:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:30 smithi143 ceph-mon[36013]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:13:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:30 smithi143 ceph-mon[36013]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:32.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 -- 172.21.15.106:0/1958623777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 msgr2=0x7f20ec10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 --2- 172.21.15.106:0/1958623777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec10a0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b0d0 tx=0x7f20e400b3e0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 -- 172.21.15.106:0/1958623777 shutdown_connections 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 --2- 172.21.15.106:0/1958623777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec10a0f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 --2- 172.21.15.106:0/1958623777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20ec101ca0 0x7f20ec1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 --2- 172.21.15.106:0/1958623777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec102ea0 0x7f20ec103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 -- 172.21.15.106:0/1958623777 >> 172.21.15.106:0/1958623777 conn(0x7f20ec0fd280 msgr2=0x7f20ec0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 -- 172.21.15.106:0/1958623777 shutdown_connections 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.396+0000 7f20f0edd700 1 -- 172.21.15.106:0/1958623777 wait complete. 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 Processor -- start 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 -- start start 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec101ca0 0x7f20ec075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20ec102ea0 0x7f20ec074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:32.401 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec074d20 con 0x7f20ec102ea0 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec074e90 con 0x7f20ec103840 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20f0edd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1a1e20 con 0x7f20ec101ca0 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56312/0 (socket says 172.21.15.106:56312) 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 -- 172.21.15.106:0/3222220627 learned_addr learned my addr 172.21.15.106:0/3222220627 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ea59c700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec101ca0 0x7f20ec075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 -- 172.21.15.106:0/3222220627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec101ca0 msgr2=0x7f20ec075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec101ca0 0x7f20ec075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 -- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20ec102ea0 msgr2=0x7f20ec074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20ec102ea0 0x7f20ec074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 -- 172.21.15.106:0/3222220627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4009d00 con 0x7f20ec103840 2024-03-11T19:13:32.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20ead9d700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f20e4007c90 tx=0x7f20e4009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:13:32.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.397+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400e040 con 0x7f20ec103840 2024-03-11T19:13:32.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.398+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20e401dd70 con 0x7f20ec103840 2024-03-11T19:13:32.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.398+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e401be60 con 0x7f20ec103840 2024-03-11T19:13:32.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.398+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec1a20a0 con 0x7f20ec103840 2024-03-11T19:13:32.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.398+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec1a2540 con 0x7f20ec103840 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.398+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec109060 con 0x7f20ec103840 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.399+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f20e401b4e0 con 0x7f20ec103840 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.400+0000 7f20db7fe700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 0x7f20d40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.400+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f20e40cd7d0 con 0x7f20ec103840 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.401+0000 7f20ea59c700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 0x7f20d40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:32.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.401+0000 7f20ea59c700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 0x7f20d40a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20ec102d00 tx=0x7f20dc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:13:32.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.404+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20e40984e0 con 0x7f20ec103840 2024-03-11T19:13:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:32.603+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f20ec02d870 con 0x7f20d40a1a30 2024-03-11T19:13:33.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:33 smithi106 ceph-mon[35889]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:33.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:33.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:33.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:33 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:33.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:33 smithi072 ceph-mon[31695]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:33.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:33 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:33.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:33 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:33.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:33 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:33 smithi143 ceph-mon[36013]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:33 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:33.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:33 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:33.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:33 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi072 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 17090M 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:34.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi072 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 17090M 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:34.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:34.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi072 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 17090M 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:35.359 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:35 smithi072 ceph-mon[31695]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:35 smithi106 ceph-mon[35889]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:35 smithi143 ceph-mon[36013]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:36.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/36236490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]': finished 2024-03-11T19:13:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:13:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:36 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/104012258' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/36236490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]': finished 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/104012258' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/36236490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]: dispatch 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7963ed34-6f58-4b33-bb70-cc1c532602f2"}]': finished 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:36 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/104012258' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:13:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:37 smithi106 ceph-mon[35889]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:37 smithi072 ceph-mon[31695]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:37 smithi143 ceph-mon[36013]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:39.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:39 smithi106 ceph-mon[35889]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:39.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:39 smithi072 ceph-mon[31695]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:39 smithi143 ceph-mon[36013]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:41.236 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:41 smithi106 ceph-mon[35889]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:41.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:41 smithi072 ceph-mon[31695]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:41.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:41 smithi143 ceph-mon[36013]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:43.144 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:43 smithi106 ceph-mon[35889]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:13:43.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:43 smithi072 ceph-mon[31695]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:13:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:43 smithi143 ceph-mon[36013]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:13:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:44 smithi106 ceph-mon[35889]: Deploying daemon osd.1 on smithi106 2024-03-11T19:13:44.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:44 smithi072 ceph-mon[31695]: Deploying daemon osd.1 on smithi106 2024-03-11T19:13:44.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:44 smithi143 ceph-mon[36013]: Deploying daemon osd.1 on smithi106 2024-03-11T19:13:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:45 smithi106 ceph-mon[35889]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:45.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:45 smithi072 ceph-mon[31695]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:45.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:45 smithi143 ceph-mon[36013]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:47 smithi106 ceph-mon[35889]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:47.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:47 smithi072 ceph-mon[31695]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:47.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:47 smithi143 ceph-mon[36013]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:48 smithi072 ceph-mon[31695]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:48 smithi143 ceph-mon[36013]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:48 smithi106 ceph-mon[35889]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.243+0000 7f20db7fe700 1 -- 172.21.15.106:0/3222220627 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f20ec02d870 con 0x7f20d40a1a30 2024-03-11T19:13:51.248 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-03-11T19:13:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 msgr2=0x7f20d40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 0x7f20d40a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20ec102d00 tx=0x7f20dc009190 comp rx=0 tx=0).stop 2024-03-11T19:13:51.249 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 msgr2=0x7f20ec0745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f20e4007c90 tx=0x7f20e4009380 comp rx=0 tx=0).stop 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 shutdown_connections 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f20d40a1a30 0x7f20d40a3ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec103840 0x7f20ec0745a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec101ca0 0x7f20ec075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 --2- 172.21.15.106:0/3222220627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20ec102ea0 0x7f20ec074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 >> 172.21.15.106:0/3222220627 conn(0x7f20ec0fd280 msgr2=0x7f20ec1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 shutdown_connections 2024-03-11T19:13:51.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:13:51.245+0000 7f20f0edd700 1 -- 172.21.15.106:0/3222220627 wait complete. 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:51.913 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.1.service 2024-03-11T19:13:51.916 INFO:tasks.cephadm:Deploying osd.2 on smithi143 with /dev/vg_nvme/lv_4... 2024-03-11T19:13:51.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:13:52.159 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:13:52.279 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.279 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.280 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:52.280 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:52 smithi143 ceph-mon[36013]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:52.280 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:52 smithi143 ceph-mon[36013]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:52.405 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.405 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.406 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:52.406 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:52 smithi106 ceph-mon[35889]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:52.406 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:52 smithi106 ceph-mon[35889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:13:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:52 smithi072 ceph-mon[31695]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:52 smithi072 ceph-mon[31695]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:13:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:13:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:13:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:53.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:53 smithi143 ceph-mon[36013]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:53.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:13:53.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:13:53.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:53.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-mon[35889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: from='osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:53 smithi072 ceph-mon[31695]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:13:53.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1[40472]: 2024-03-11T19:13:53.675+0000 7f3349327700 -1 osd.1 0 waiting for initial osdmap 2024-03-11T19:13:53.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:13:53 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1[40472]: 2024-03-11T19:13:53.679+0000 7f3340911700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126514 s, 829 MB/s 2024-03-11T19:13:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: from='osd.1 ' entity='osd.1' 2024-03-11T19:13:54.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:54 smithi072 ceph-mon[31695]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:54.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:13:54.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:13:54.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.638 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: from='osd.1 ' entity='osd.1' 2024-03-11T19:13:54.638 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:54 smithi143 ceph-mon[36013]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: from='osd.1 ' entity='osd.1' 2024-03-11T19:13:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:54 smithi106 ceph-mon[35889]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:13:54.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-03-11T19:13:54.895 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] boot 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi106 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.349 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:55.350 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-11T19:13:55.350 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.350 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:55.350 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:55.350 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] boot 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi106 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-11T19:13:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: osd.1 [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] boot 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi106 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:13:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:13:56.266 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:56 smithi143 ceph-mon[36013]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 -- 172.21.15.143:0/3635587692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc101ca0 msgr2=0x7fe9cc1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/3635587692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc1020c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009990 tx=0x7fe9bc009ca0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 -- 172.21.15.143:0/3635587692 shutdown_connections 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/3635587692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc103840 0x7fe9cc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/3635587692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc1020c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/3635587692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 -- 172.21.15.143:0/3635587692 >> 172.21.15.143:0/3635587692 conn(0x7fe9cc0fd280 msgr2=0x7fe9cc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 -- 172.21.15.143:0/3635587692 shutdown_connections 2024-03-11T19:13:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 -- 172.21.15.143:0/3635587692 wait complete. 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.504+0000 7fe9d20d0700 1 Processor -- start 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- start start 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9cc103840 0x7fe9cc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc19f980 con 0x7fe9cc103840 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc19fac0 con 0x7fe9cc101ca0 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc19fc50 con 0x7fe9cc102ea0 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38546/0 (socket says 172.21.15.143:38546) 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 -- 172.21.15.143:0/102815917 learned_addr learned my addr 172.21.15.143:0/102815917 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:13:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9cb7fe700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 -- 172.21.15.143:0/102815917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 msgr2=0x7fe9cc197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 -- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9cc103840 msgr2=0x7fe9cc19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9cc103840 0x7fe9cc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 -- 172.21.15.143:0/102815917 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0095f0 con 0x7fe9cc102ea0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9cb7fe700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c3fff700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400b560 tx=0x7fe9b400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4011830 con 0x7fe9cc102ea0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9b4011e70 con 0x7fe9cc102ea0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc19ff30 con 0x7fe9cc102ea0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400f2e0 con 0x7fe9cc102ea0 2024-03-11T19:13:56.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.505+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1a0470 con 0x7fe9cc102ea0 2024-03-11T19:13:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.506+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc068930 con 0x7fe9cc102ea0 2024-03-11T19:13:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.507+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7fe9b4011990 con 0x7fe9cc102ea0 2024-03-11T19:13:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.508+0000 7fe9c97fa700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 0x7fe9b80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:13:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.508+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fe9b40cb620 con 0x7fe9cc102ea0 2024-03-11T19:13:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.508+0000 7fe9cb7fe700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 0x7fe9b80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:13:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.509+0000 7fe9cb7fe700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 0x7fe9b80a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc003930 tx=0x7fe9bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:13:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.513+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9b4096080 con 0x7fe9cc102ea0 2024-03-11T19:13:56.632 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:56 smithi072 ceph-mon[31695]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:56.644 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:56 smithi106 ceph-mon[35889]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:56.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:13:56.717+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9cc02d910 con 0x7fe9b80a1ab0 2024-03-11T19:13:57.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:57 smithi143 ceph-mon[36013]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:13:57.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:57 smithi143 ceph-mon[36013]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:57.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:57.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:57.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:57 smithi106 ceph-mon[35889]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:13:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:57 smithi106 ceph-mon[35889]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:57 smithi072 ceph-mon[31695]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:13:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:57 smithi072 ceph-mon[31695]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:13:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:13:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:13:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:13:58.546 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:58 smithi143 ceph-mon[36013]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:58.644 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:58 smithi106 ceph-mon[35889]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:58.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:58 smithi072 ceph-mon[31695]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:13:59.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:59 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/4247858916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:59 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:59 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]': finished 2024-03-11T19:13:59.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:59 smithi143 ceph-mon[36013]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:13:59.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:13:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:13:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:59 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/4247858916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:59 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:59 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]': finished 2024-03-11T19:13:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:59 smithi106 ceph-mon[35889]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:13:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:13:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:13:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:59 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/4247858916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:59 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]: dispatch 2024-03-11T19:13:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:59 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "843038a5-98bb-4739-8a50-a0ef1fe16d3d"}]': finished 2024-03-11T19:13:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:59 smithi072 ceph-mon[31695]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:13:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:13:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:00.644 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/2148721729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:00 smithi106 ceph-mon[35889]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:00.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/2148721729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:00.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:00 smithi072 ceph-mon[31695]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/2148721729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:00 smithi143 ceph-mon[36013]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:03.133 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:03 smithi143 ceph-mon[36013]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:03.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:03 smithi106 ceph-mon[35889]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:03.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:03 smithi072 ceph-mon[31695]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:05.225 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:05 smithi143 ceph-mon[36013]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:05 smithi106 ceph-mon[35889]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:05.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:05 smithi072 ceph-mon[31695]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:07.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:07 smithi143 ceph-mon[36013]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:07.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:14:07.140 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:07 smithi106 ceph-mon[35889]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:14:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:07.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:07 smithi072 ceph-mon[31695]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:14:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:08.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:08 smithi143 ceph-mon[36013]: Deploying daemon osd.2 on smithi143 2024-03-11T19:14:08.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:08 smithi106 ceph-mon[35889]: Deploying daemon osd.2 on smithi143 2024-03-11T19:14:08.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:08 smithi072 ceph-mon[31695]: Deploying daemon osd.2 on smithi143 2024-03-11T19:14:09.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:09 smithi143 ceph-mon[36013]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:09.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:09 smithi106 ceph-mon[35889]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:09.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:09 smithi072 ceph-mon[31695]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:11 smithi106 ceph-mon[35889]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:11 smithi072 ceph-mon[31695]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:11 smithi143 ceph-mon[36013]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:12 smithi106 ceph-mon[35889]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:12 smithi072 ceph-mon[31695]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:12 smithi143 ceph-mon[36013]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.560+0000 7fe9c97fa700 1 -- 172.21.15.143:0/102815917 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe9cc02d910 con 0x7fe9b80a1ab0 2024-03-11T19:14:14.564 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 2 on host 'smithi143' 2024-03-11T19:14:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 msgr2=0x7fe9b80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 0x7fe9b80a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc003930 tx=0x7fe9bc005fb0 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 msgr2=0x7fe9cc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400b560 tx=0x7fe9b400b920 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 shutdown_connections 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe9b80a1ab0 0x7fe9b80a3f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9cc103840 0x7fe9cc19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9cc101ca0 0x7fe9cc197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 --2- 172.21.15.143:0/102815917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9cc102ea0 0x7fe9cc198330 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.562+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 >> 172.21.15.143:0/102815917 conn(0x7fe9cc0fd280 msgr2=0x7fe9cc1062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.563+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 shutdown_connections 2024-03-11T19:14:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:14:14.563+0000 7fe9d20d0700 1 -- 172.21.15.143:0/102815917 wait complete. 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:15.209 DEBUG:teuthology.orchestra.run.smithi143:osd.2> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.2.service 2024-03-11T19:14:15.211 INFO:tasks.cephadm:Deploying osd.3 on smithi072 with /dev/vg_nvme/lv_3... 2024-03-11T19:14:15.212 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:15.662 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:14:16.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:16 smithi106 ceph-mon[35889]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:16.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:16 smithi106 ceph-mon[35889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:16 smithi072 ceph-mon[31695]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:16.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:16 smithi072 ceph-mon[31695]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:16.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:16 smithi143 ceph-mon[36013]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:16.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:16 smithi143 ceph-mon[36013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:14:17.371 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:17.371 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:14:17.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:14:17.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:17.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:17 smithi072 ceph-mon[31695]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:17 smithi106 ceph-mon[35889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: from='osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-mon[36013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:14:17.922 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2[41124]: 2024-03-11T19:14:17.531+0000 7fe0d8884700 -1 osd.2 0 waiting for initial osdmap 2024-03-11T19:14:17.922 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2[41124]: 2024-03-11T19:14:17.536+0000 7fe0d066f700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:14:18.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:18 smithi143 ceph-mon[36013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:14:18.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:18 smithi143 ceph-mon[36013]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:14:18.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:18.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:18 smithi143 ceph-mon[36013]: from='osd.2 ' entity='osd.2' 2024-03-11T19:14:18.392 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:18 smithi072 ceph-mon[31695]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:14:18.392 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:18 smithi072 ceph-mon[31695]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:14:18.392 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:18.392 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:18 smithi072 ceph-mon[31695]: from='osd.2 ' entity='osd.2' 2024-03-11T19:14:18.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:18 smithi106 ceph-mon[35889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:14:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:18 smithi106 ceph-mon[35889]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:14:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:18 smithi106 ceph-mon[35889]: from='osd.2 ' entity='osd.2' 2024-03-11T19:14:18.601 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:14:18.601 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:14:18.601 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-03-11T19:14:18.602 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-03-11T19:14:18.602 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0582959 s, 180 MB/s 2024-03-11T19:14:18.602 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] boot 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:14:19.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:14:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] boot 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:14:19.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:19.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: osd.2 [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] boot 2024-03-11T19:14:19.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:14:19.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:14:19.726 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-03-11T19:14:20.138 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:14:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:20 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi143 2024-03-11T19:14:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:20 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 17080M 2024-03-11T19:14:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:20 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi143 2024-03-11T19:14:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:20 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 17080M 2024-03-11T19:14:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:20 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi143 2024-03-11T19:14:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:20 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 17080M 2024-03-11T19:14:21.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:21 smithi106 ceph-mon[35889]: pgmap v93: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:21.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:21.899 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:21.899 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:21 smithi106 ceph-mon[35889]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:14:21.899 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:21 smithi072 ceph-mon[31695]: pgmap v93: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:21 smithi072 ceph-mon[31695]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:14:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:21 smithi143 ceph-mon[36013]: pgmap v93: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:21 smithi143 ceph-mon[36013]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:14:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:14:22.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 -- 172.21.15.72:0/1326441383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 msgr2=0x7f6c981009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 --2- 172.21.15.72:0/1326441383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c981009a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009990 tx=0x7f6c88009ca0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 -- 172.21.15.72:0/1326441383 shutdown_connections 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 --2- 172.21.15.72:0/1326441383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c98101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 --2- 172.21.15.72:0/1326441383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c981009a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 --2- 172.21.15.72:0/1326441383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 0x7f6c98101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 -- 172.21.15.72:0/1326441383 >> 172.21.15.72:0/1326441383 conn(0x7f6c980fc140 msgr2=0x7f6c980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 -- 172.21.15.72:0/1326441383 shutdown_connections 2024-03-11T19:14:22.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.476+0000 7f6c9d800700 1 -- 172.21.15.72:0/1326441383 wait complete. 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.477+0000 7f6c9d800700 1 Processor -- start 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.477+0000 7f6c9d800700 1 -- start start 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 0x7f6c9819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c981a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981a19d0 con 0x7f6c981018a0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981a1b10 con 0x7f6c98100580 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c9d800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981a1ca0 con 0x7f6c98100ee0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:53980/0 (socket says 172.21.15.72:53980) 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 -- 172.21.15.72:0/2139035372 learned_addr learned my addr 172.21.15.72:0/2139035372 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96000700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 0x7f6c9819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c97400700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c981a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 -- 172.21.15.72:0/2139035372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 msgr2=0x7f6c9819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 0x7f6c9819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 -- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 msgr2=0x7f6c981a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c981a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c96a00700 1 -- 172.21.15.72:0/2139035372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880095f0 con 0x7f6c98100580 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.478+0000 7f6c97400700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c981a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c96a00700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c010 tx=0x7f6c880057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:14:22.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8801d070 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c8800fb40 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c981a1f20 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88017650 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.479+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981a2280 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.480+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9810b360 con 0x7f6c98100580 2024-03-11T19:14:22.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.482+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f6c8800fcb0 con 0x7f6c98100580 2024-03-11T19:14:22.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.483+0000 7f6c8f400700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 0x7f6c7c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:22.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.483+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f6c880cd9d0 con 0x7f6c98100580 2024-03-11T19:14:22.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.483+0000 7f6c96000700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 0x7f6c7c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:22.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.484+0000 7f6c96000700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 0x7f6c7c0a3ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009bf0 tx=0x7f6c84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:14:22.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.489+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c88098080 con 0x7f6c98100580 2024-03-11T19:14:22.624 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:22.625 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:22.625 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50634]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:22.625 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50634]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.627+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6c88098450 con 0x7f6c98100580 2024-03-11T19:14:22.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:22.730+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c9802d950 con 0x7f6c7c0a1a30 2024-03-11T19:14:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:22 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:22 smithi106 ceph-mon[35889]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:14:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:22 smithi106 ceph-mon[35889]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:22.902 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:22 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:22.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:22 smithi072 ceph-mon[31695]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:14:22.903 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:22 smithi072 ceph-mon[31695]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:22 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:14:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:22 smithi143 ceph-mon[36013]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:14:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:22 smithi143 ceph-mon[36013]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:23.180 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:23.180 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:23.180 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50718]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:23.180 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:14:22 smithi072 sudo[50718]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:23.394 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44296]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:23.395 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:23.395 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44296]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:23.395 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44296]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:23.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:23.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:23.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:23.895 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:14:23 smithi106 sudo[44299]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:23.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:14:23.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:14:23.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:23.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:23.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:23.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:23.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:23.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:23.925 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:23.926 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:23.939 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:23.939 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:23.939 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44780]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:23.939 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44780]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:23.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:23.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:24.439 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:24.439 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:24.439 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44783]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:24.439 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:14:23 smithi143 sudo[44783]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:24.900 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:24 smithi072 sudo[50806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:24.900 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:24 smithi072 sudo[50806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:24.900 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:24 smithi072 sudo[50806]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:24.901 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:24 smithi072 sudo[50806]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:25.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 sudo[50830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:25.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 sudo[50830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:25.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 sudo[50830]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:25.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 sudo[50830]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:25.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:25.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:25.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44786]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:25.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44786]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:25.859 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:25.859 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:25.859 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 sudo[44302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:14:25.860 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 sudo[44302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:25.860 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 sudo[44302]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:25.860 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:25 smithi106 sudo[44302]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:25 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44789]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 sudo[44789]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:25 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 sudo[44305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:14:26.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 sudo[44305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:14:26.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 sudo[44305]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:14:26.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 sudo[44305]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:26 smithi106 ceph-mon[35889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:26 smithi072 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:26 smithi143 ceph-mon[36013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1121831543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]': finished 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:27 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3579219922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1121831543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]': finished 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:27 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3579219922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1121831543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]: dispatch 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59081b5d-c259-41da-89c9-0e99ed857bed"}]': finished 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:27 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3579219922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:14:28.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:28 smithi106 ceph-mon[35889]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:28.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:28 smithi072 ceph-mon[31695]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:28 smithi143 ceph-mon[36013]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:31.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:31 smithi072 ceph-mon[31695]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:31.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:31 smithi106 ceph-mon[35889]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:31.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:31 smithi143 ceph-mon[36013]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:33.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:33 smithi072 ceph-mon[31695]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-11T19:14:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:33 smithi106 ceph-mon[35889]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-11T19:14:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:33 smithi143 ceph-mon[36013]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-11T19:14:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:35 smithi106 ceph-mon[35889]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-11T19:14:35.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:35 smithi072 ceph-mon[31695]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-11T19:14:35.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:35 smithi143 ceph-mon[36013]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-11T19:14:37.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:37 smithi106 ceph-mon[35889]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:14:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:14:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:37 smithi072 ceph-mon[31695]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:14:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:14:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:37 smithi143 ceph-mon[36013]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:14:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:14:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:38.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:38 smithi072 ceph-mon[31695]: Deploying daemon osd.3 on smithi072 2024-03-11T19:14:38.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:38 smithi106 ceph-mon[35889]: Deploying daemon osd.3 on smithi072 2024-03-11T19:14:38.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:38 smithi143 ceph-mon[36013]: Deploying daemon osd.3 on smithi072 2024-03-11T19:14:39.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:39 smithi072 ceph-mon[31695]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 907 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-11T19:14:39.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:39 smithi106 ceph-mon[35889]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 907 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-11T19:14:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:39 smithi143 ceph-mon[36013]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 907 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-11T19:14:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:41 smithi072 ceph-mon[31695]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:41.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:41 smithi106 ceph-mon[35889]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:41 smithi143 ceph-mon[36013]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:43.349 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:43 smithi072 ceph-mon[31695]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:43 smithi106 ceph-mon[35889]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:43 smithi143 ceph-mon[36013]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-11T19:14:45.389 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:45 smithi072 ceph-mon[31695]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:45.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:45 smithi106 ceph-mon[35889]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:45 smithi143 ceph-mon[36013]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:46.376 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.376 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.376 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:46.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:47 smithi072 ceph-mon[31695]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:47.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:47 smithi106 ceph-mon[35889]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:47 smithi143 ceph-mon[36013]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:49 smithi072 ceph-mon[31695]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:49 smithi106 ceph-mon[35889]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:14:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:49.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:49.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:49 smithi143 ceph-mon[36013]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:14:50.077 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 3 on host 'smithi072' 2024-03-11T19:14:50.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.073+0000 7f6c8f400700 1 -- 172.21.15.72:0/2139035372 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c9802d950 con 0x7f6c7c0a1a30 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 msgr2=0x7f6c7c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 0x7f6c7c0a3ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009bf0 tx=0x7f6c84009260 comp rx=0 tx=0).stop 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 msgr2=0x7f6c98199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c010 tx=0x7f6c880057f0 comp rx=0 tx=0).stop 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 shutdown_connections 2024-03-11T19:14:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6c7c0a1a30 0x7f6c7c0a3ef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c981018a0 0x7f6c981a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c98100580 0x7f6c98199e90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 --2- 172.21.15.72:0/2139035372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98100ee0 0x7f6c9819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 >> 172.21.15.72:0/2139035372 conn(0x7f6c980fc140 msgr2=0x7f6c980fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.077+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 shutdown_connections 2024-03-11T19:14:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:14:50.078+0000 7f6c9d800700 1 -- 172.21.15.72:0/2139035372 wait complete. 2024-03-11T19:14:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:14:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:14:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:14:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:50.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:50.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:50.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:51.063 DEBUG:teuthology.orchestra.run.smithi072:osd.3> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.3.service 2024-03-11T19:14:51.067 INFO:tasks.cephadm:Deploying osd.4 on smithi106 with /dev/vg_nvme/lv_3... 2024-03-11T19:14:51.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:14:51.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:51.098 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:14:51.320 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:14:51.679 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:14:51.423+0000 7f2d4fc00700 -1 osd.3 0 waiting for initial osdmap 2024-03-11T19:14:51.680 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:14:51.459+0000 7f2d45a00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: osdmap e25: 4 total, 3 up, 4 in 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:51 smithi106 ceph-mon[35889]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: osdmap e25: 4 total, 3 up, 4 in 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:51 smithi072 ceph-mon[31695]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: osdmap e25: 4 total, 3 up, 4 in 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:51 smithi143 ceph-mon[36013]: from='osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755]' entity='osd.3' 2024-03-11T19:14:52.919 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] boot 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:52.920 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:52 smithi106 ceph-mon[35889]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:53.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:14:53.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:14:53.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:53.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] boot 2024-03-11T19:14:53.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:14:53.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:53.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:52 smithi072 ceph-mon[31695]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: osd.3 [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] boot 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:14:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:52 smithi143 ceph-mon[36013]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:14:53.370 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:14:53.370 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:14:53.370 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-11T19:14:53.370 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-11T19:14:53.370 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122172 s, 858 MB/s 2024-03-11T19:14:53.371 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-11T19:14:54.032 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:53 smithi106 ceph-mon[35889]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:14:54.059 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-03-11T19:14:54.123 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:53 smithi072 ceph-mon[31695]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:14:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:53 smithi143 ceph-mon[36013]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:14:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:14:55.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:54 smithi106 ceph-mon[35889]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:14:55.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:54 smithi106 ceph-mon[35889]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:14:55.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:54 smithi072 ceph-mon[31695]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:14:55.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:54 smithi072 ceph-mon[31695]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:14:55.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:54 smithi143 ceph-mon[36013]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:14:55.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:54 smithi143 ceph-mon[36013]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 -- 172.21.15.106:0/2651024890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 msgr2=0x7f5cc0108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 --2- 172.21.15.106:0/2651024890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc0108790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b0d0 tx=0x7f5cb800b3e0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 -- 172.21.15.106:0/2651024890 shutdown_connections 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 --2- 172.21.15.106:0/2651024890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc0108790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 --2- 172.21.15.106:0/2651024890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0100340 0x7f5cc0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 --2- 172.21.15.106:0/2651024890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0101540 0x7f5cc01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 -- 172.21.15.106:0/2651024890 >> 172.21.15.106:0/2651024890 conn(0x7f5cc00fb8c0 msgr2=0x7f5cc00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 -- 172.21.15.106:0/2651024890 shutdown_connections 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.934+0000 7f5cc8414700 1 -- 172.21.15.106:0/2651024890 wait complete. 2024-03-11T19:14:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 Processor -- start 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 -- start start 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 0x7f5cc0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 0x7f5cc0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc019f970 con 0x7f5cc0101540 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc019fab0 con 0x7f5cc0101ee0 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc8414700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc019fc40 con 0x7f5cc0100340 2024-03-11T19:14:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57974/0 (socket says 172.21.15.106:57974) 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 -- 172.21.15.106:0/285789773 learned_addr learned my addr 172.21.15.106:0/285789773 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc59af700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 0x7f5cc0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 -- 172.21.15.106:0/285789773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 msgr2=0x7f5cc0197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc61b0700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 0x7f5cc0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 0x7f5cc0197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 -- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 msgr2=0x7f5cc0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 0x7f5cc0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:14:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc69b1700 1 -- 172.21.15.106:0/285789773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8009d00 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.935+0000 7f5cc61b0700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 0x7f5cc0197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cc59af700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 0x7f5cc0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cc69b1700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8007c90 tx=0x7f5cb8009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb800e040 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8004e60 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc019fec0 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8012bf0 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.936+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc01a0350 con 0x7f5cc0101ee0 2024-03-11T19:14:55.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.937+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc0068930 con 0x7f5cc0101ee0 2024-03-11T19:14:55.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.938+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5cb8012d50 con 0x7f5cc0101ee0 2024-03-11T19:14:55.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.938+0000 7f5cb77fe700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 0x7f5cac0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:14:55.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.939+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f5cb80cd970 con 0x7f5cc0101ee0 2024-03-11T19:14:55.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.939+0000 7f5cc61b0700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 0x7f5cac0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:14:55.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.939+0000 7f5cc61b0700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 0x7f5cac0a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0005fd0 tx=0x7f5cb0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:14:55.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:55.942+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cb8097e30 con 0x7f5cc0101ee0 2024-03-11T19:14:56.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:55 smithi106 ceph-mon[35889]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:14:56.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:14:56.142+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cc0107990 con 0x7f5cac0a1b40 2024-03-11T19:14:56.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:55 smithi072 ceph-mon[31695]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:14:56.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:55 smithi143 ceph-mon[36013]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: osdmap e30: 4 total, 4 up, 4 in 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:56.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:56 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: osdmap e30: 4 total, 4 up, 4 in 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:57.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:56 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: osdmap e30: 4 total, 4 up, 4 in 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:14:57.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:56 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi072 2024-03-11T19:14:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 8545M 2024-03-11T19:14:58.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:58.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:58.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi072 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 8545M 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:58.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:58.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi072 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 8545M 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:14:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/3234638008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]': finished 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:14:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:14:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/3234638008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]': finished 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:14:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:14:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/3234638008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]: dispatch 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f"}]': finished 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:14:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:14:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:00.644 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/2527009222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:00.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/2527009222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/2527009222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:01 smithi106 ceph-mon[35889]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:15:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:01 smithi072 ceph-mon[31695]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:15:01.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:01 smithi143 ceph-mon[36013]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:15:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:03 smithi106 ceph-mon[35889]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:15:03.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:03 smithi072 ceph-mon[31695]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:15:03.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:03 smithi143 ceph-mon[36013]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:15:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:05 smithi106 ceph-mon[35889]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:05.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:05 smithi072 ceph-mon[31695]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:05 smithi143 ceph-mon[36013]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:06 smithi106 ceph-mon[35889]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:15:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:06 smithi106 ceph-mon[35889]: Deploying daemon osd.4 on smithi106 2024-03-11T19:15:06.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:06 smithi072 ceph-mon[31695]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:06.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:15:06.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:06.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:06 smithi072 ceph-mon[31695]: Deploying daemon osd.4 on smithi106 2024-03-11T19:15:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:06 smithi143 ceph-mon[36013]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:15:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:06 smithi143 ceph-mon[36013]: Deploying daemon osd.4 on smithi106 2024-03-11T19:15:09.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:09 smithi106 ceph-mon[35889]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:09.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:09 smithi072 ceph-mon[31695]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:09 smithi143 ceph-mon[36013]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:11.219 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:11 smithi106 ceph-mon[35889]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:11 smithi072 ceph-mon[31695]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:11 smithi143 ceph-mon[36013]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:13.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:13.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:12 smithi106 ceph-mon[35889]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:13.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:13.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:12 smithi072 ceph-mon[31695]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:13.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:13.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:13.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:12 smithi143 ceph-mon[36013]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:15.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:15 smithi106 ceph-mon[35889]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:15.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.273+0000 7f5cb77fe700 1 -- 172.21.15.106:0/285789773 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5cc0107990 con 0x7f5cac0a1b40 2024-03-11T19:15:15.277 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 4 on host 'smithi106' 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 msgr2=0x7f5cac0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 0x7f5cac0a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0005fd0 tx=0x7f5cb0005dc0 comp rx=0 tx=0).stop 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 msgr2=0x7f5cc019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8007c90 tx=0x7f5cb8009430 comp rx=0 tx=0).stop 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 shutdown_connections 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5cac0a1b40 0x7f5cac0a4000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc0101ee0 0x7f5cc019f3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:15.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cc0100340 0x7f5cc0197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:15.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 --2- 172.21.15.106:0/285789773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cc0101540 0x7f5cc0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:15.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 >> 172.21.15.106:0/285789773 conn(0x7f5cc00fb8c0 msgr2=0x7f5cc00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:15:15.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 shutdown_connections 2024-03-11T19:15:15.280 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:15:15.275+0000 7f5cc8414700 1 -- 172.21.15.106:0/285789773 wait complete. 2024-03-11T19:15:15.380 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:15 smithi143 ceph-mon[36013]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:15 smithi072 ceph-mon[31695]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:16.001 DEBUG:teuthology.orchestra.run.smithi106:osd.4> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.4.service 2024-03-11T19:15:16.003 INFO:tasks.cephadm:Deploying osd.5 on smithi143 with /dev/vg_nvme/lv_3... 2024-03-11T19:15:16.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:15:16.242 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.364 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.365 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:16 smithi143 ceph-mon[36013]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:16 smithi106 ceph-mon[35889]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:15:16.707 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:16 smithi072 ceph-mon[31695]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-mon[35889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:15:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-mon[35889]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-mon[35889]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:15:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-mon[35889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:17.661 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:17 smithi143 ceph-mon[36013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:15:17.661 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:17 smithi143 ceph-mon[36013]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:17.661 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:17 smithi143 ceph-mon[36013]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:15:17.661 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:17.661 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:17 smithi143 ceph-mon[36013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:17 smithi072 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:15:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:17 smithi072 ceph-mon[31695]: from='osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:17 smithi072 ceph-mon[31695]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:15:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:17 smithi072 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:15:18.028 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:15:17.823+0000 7f1b9a437700 -1 osd.4 0 waiting for initial osdmap 2024-03-11T19:15:18.028 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:15:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:15:17.828+0000 7f1b91a21700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125502 s, 836 MB/s 2024-03-11T19:15:18.312 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-03-11T19:15:18.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:15:18.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: osdmap e33: 5 total, 4 up, 5 in 2024-03-11T19:15:18.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: from='osd.4 ' entity='osd.4' 2024-03-11T19:15:18.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:18 smithi143 ceph-mon[36013]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: osdmap e33: 5 total, 4 up, 5 in 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: from='osd.4 ' entity='osd.4' 2024-03-11T19:15:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:18 smithi072 ceph-mon[31695]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:18.809 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:15:18.809 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: osdmap e33: 5 total, 4 up, 5 in 2024-03-11T19:15:18.810 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.810 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:18.810 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: from='osd.4 ' entity='osd.4' 2024-03-11T19:15:18.810 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:18 smithi106 ceph-mon[35889]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:15:19.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-03-11T19:15:19.271 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] boot 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:15:19.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:19.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:15:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:15:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] boot 2024-03-11T19:15:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:15:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: osd.4 [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] boot 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:15:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:15:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi106 2024-03-11T19:15:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.691 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:20 smithi143 ceph-mon[36013]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi106 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:20 smithi106 ceph-mon[35889]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi106 2024-03-11T19:15:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:20.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:20 smithi072 ceph-mon[31695]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:21.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 -- 172.21.15.143:0/3052110498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac1039e0 msgr2=0x7f71ac10a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:21.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 --2- 172.21.15.143:0/3052110498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac1039e0 0x7f71ac10a290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b0a0 tx=0x7f71a400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 -- 172.21.15.143:0/3052110498 shutdown_connections 2024-03-11T19:15:21.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 --2- 172.21.15.143:0/3052110498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac1039e0 0x7f71ac10a290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 --2- 172.21.15.143:0/3052110498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71ac0fffc0 0x7f71ac1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 --2- 172.21.15.143:0/3052110498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac103040 0x7f71ac1034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 -- 172.21.15.143:0/3052110498 >> 172.21.15.143:0/3052110498 conn(0x7f71ac0fb900 msgr2=0x7f71ac0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 -- 172.21.15.143:0/3052110498 shutdown_connections 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.997+0000 7f71b23f6700 1 -- 172.21.15.143:0/3052110498 wait complete. 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 Processor -- start 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 -- start start 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71ac0fffc0 0x7f71ac195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 0x7f71ac1ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19b320 con 0x7f71ac0fffc0 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19b490 con 0x7f71ac1039e0 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71b23f6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19b650 con 0x7f71ac103040 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:21.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52470/0 (socket says 172.21.15.143:52470) 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.998+0000 7f71ab7fe700 1 -- 172.21.15.143:0/4041679081 learned_addr learned my addr 172.21.15.143:0/4041679081 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71b0993700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 0x7f71ac1ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 -- 172.21.15.143:0/4041679081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 msgr2=0x7f71ac1ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 0x7f71ac1ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 -- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71ac0fffc0 msgr2=0x7f71ac195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71ac0fffc0 0x7f71ac195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 -- 172.21.15.143:0/4041679081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009d00 con 0x7f71ac103040 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71b0993700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 0x7f71ac1ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71ab7fe700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b530 tx=0x7f71a000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0011830 con 0x7f71ac103040 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac1aee50 con 0x7f71ac103040 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71a0011e70 con 0x7f71ac103040 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000f2e0 con 0x7f71ac103040 2024-03-11T19:15:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:20.999+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac1af390 con 0x7f71ac103040 2024-03-11T19:15:21.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.000+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac107120 con 0x7f71ac103040 2024-03-11T19:15:21.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.001+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f71a00105f0 con 0x7f71ac103040 2024-03-11T19:15:21.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.002+0000 7f71a97fa700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 0x7f71940a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:21.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.002+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f71a00cc2e0 con 0x7f71ac103040 2024-03-11T19:15:21.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.002+0000 7f71abfff700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 0x7f71940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:21.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.003+0000 7f71abfff700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 0x7f71940a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71ac06a0f0 tx=0x7f719c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:15:21.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.005+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71a0096560 con 0x7f71ac103040 2024-03-11T19:15:21.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:21.211+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f71ac02d8e0 con 0x7f71940a1ac0 2024-03-11T19:15:21.842 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:21 smithi143 ceph-mon[36013]: osdmap e35: 5 total, 5 up, 5 in 2024-03-11T19:15:21.842 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:21 smithi143 ceph-mon[36013]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:21.842 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:21.843 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:21.843 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:21 smithi106 ceph-mon[35889]: osdmap e35: 5 total, 5 up, 5 in 2024-03-11T19:15:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:21 smithi106 ceph-mon[35889]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:21.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:21.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:21 smithi072 ceph-mon[31695]: osdmap e35: 5 total, 5 up, 5 in 2024-03-11T19:15:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:21 smithi072 ceph-mon[31695]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:22.862 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:22 smithi143 ceph-mon[36013]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:22.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:22 smithi106 ceph-mon[35889]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:22 smithi072 ceph-mon[31695]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:24.352 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:24 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/1236680991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.352 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:24 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.352 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:24 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]': finished 2024-03-11T19:15:24.352 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:24 smithi143 ceph-mon[36013]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:15:24.352 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:24 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:24 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/1236680991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:24 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:24 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]': finished 2024-03-11T19:15:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:24 smithi106 ceph-mon[35889]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:15:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:24 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:24 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/1236680991' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:24 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]: dispatch 2024-03-11T19:15:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:24 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "316bc6e8-8a64-4578-9251-779c72149fbe"}]': finished 2024-03-11T19:15:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:24 smithi072 ceph-mon[31695]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:15:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:24 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:25 smithi106 ceph-mon[35889]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:25 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/638735806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:25 smithi072 ceph-mon[31695]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:25 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/638735806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:25 smithi143 ceph-mon[36013]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:25 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/638735806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:27 smithi106 ceph-mon[35889]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:27.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:27 smithi072 ceph-mon[31695]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:27.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:27 smithi143 ceph-mon[36013]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:29.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:29 smithi143 ceph-mon[36013]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:29.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:29 smithi106 ceph-mon[35889]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:29.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:29 smithi072 ceph-mon[31695]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:31 smithi106 ceph-mon[35889]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:31.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:31 smithi143 ceph-mon[36013]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:31.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:31 smithi072 ceph-mon[31695]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:32.326 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:32 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:15:32.326 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:32 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:32.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:32 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:15:32.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:32 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:32 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:15:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:32 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:33.239 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:33 smithi143 ceph-mon[36013]: Deploying daemon osd.5 on smithi143 2024-03-11T19:15:33.239 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:33 smithi143 ceph-mon[36013]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:33 smithi106 ceph-mon[35889]: Deploying daemon osd.5 on smithi143 2024-03-11T19:15:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:33 smithi106 ceph-mon[35889]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:33.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:33 smithi072 ceph-mon[31695]: Deploying daemon osd.5 on smithi143 2024-03-11T19:15:33.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:33 smithi072 ceph-mon[31695]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:35.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:35 smithi143 ceph-mon[36013]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:35 smithi106 ceph-mon[35889]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:35.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:35 smithi072 ceph-mon[31695]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:37 smithi106 ceph-mon[35889]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:37 smithi072 ceph-mon[31695]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:37 smithi143 ceph-mon[36013]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:38.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:38.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:38.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:38.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:38.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:38.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:38.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:38.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:38.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:39.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:39 smithi106 ceph-mon[35889]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:39 smithi072 ceph-mon[31695]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:39 smithi143 ceph-mon[36013]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.945+0000 7f71a97fa700 1 -- 172.21.15.143:0/4041679081 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71ac02d8e0 con 0x7f71940a1ac0 2024-03-11T19:15:40.949 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 5 on host 'smithi143' 2024-03-11T19:15:40.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 msgr2=0x7f71940a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 0x7f71940a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71ac06a0f0 tx=0x7f719c006d20 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 msgr2=0x7f71ac196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b530 tx=0x7f71a000b8f0 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 shutdown_connections 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71940a1ac0 0x7f71940a3f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71ac1039e0 0x7f71ac1ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71ac0fffc0 0x7f71ac195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 --2- 172.21.15.143:0/4041679081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71ac103040 0x7f71ac196220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 >> 172.21.15.143:0/4041679081 conn(0x7f71ac0fb900 msgr2=0x7f71ac104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 shutdown_connections 2024-03-11T19:15:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:15:40.947+0000 7f71b23f6700 1 -- 172.21.15.143:0/4041679081 wait complete. 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:41.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:41.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:40 smithi143 ceph-mon[36013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:40 smithi106 ceph-mon[35889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:15:41.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:40 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:41.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:41 smithi072 ceph-mon[31695]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:41 smithi072 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:15:41.605 DEBUG:teuthology.orchestra.run.smithi143:osd.5> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.5.service 2024-03-11T19:15:41.608 INFO:tasks.cephadm:Deploying osd.6 on smithi072 with /dev/vg_nvme/lv_2... 2024-03-11T19:15:41.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:15:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:15:43.095 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:15:43.095 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.095 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:15:43.095 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:43.096 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.096 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:42 smithi143 ceph-mon[36013]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:43.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:15:43.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:15:43.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:43.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:42 smithi072 ceph-mon[31695]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: from='osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:15:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:42 smithi106 ceph-mon[35889]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:15:43.689 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:15:43.373+0000 7f46a1547700 -1 osd.5 0 waiting for initial osdmap 2024-03-11T19:15:43.689 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:15:43.383+0000 7f4699332700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:15:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-mon[36013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:15:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-mon[36013]: osdmap e38: 6 total, 5 up, 6 in 2024-03-11T19:15:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:43 smithi143 ceph-mon[36013]: from='osd.5 ' entity='osd.5' 2024-03-11T19:15:44.242 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:43 smithi072 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:15:44.242 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:43 smithi072 ceph-mon[31695]: osdmap e38: 6 total, 5 up, 6 in 2024-03-11T19:15:44.242 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.242 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.242 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:43 smithi072 ceph-mon[31695]: from='osd.5 ' entity='osd.5' 2024-03-11T19:15:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:43 smithi106 ceph-mon[35889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:15:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:43 smithi106 ceph-mon[35889]: osdmap e38: 6 total, 5 up, 6 in 2024-03-11T19:15:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:43 smithi106 ceph-mon[35889]: from='osd.5 ' entity='osd.5' 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] boot 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:45.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0336545 s, 312 MB/s 2024-03-11T19:15:45.195 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-03-11T19:15:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:15:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:15:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] boot 2024-03-11T19:15:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:15:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:15:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: osd.5 [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] boot 2024-03-11T19:15:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:15:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:15:45.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:15:46.100 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:45 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi143 2024-03-11T19:15:46.100 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:45 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 8540M 2024-03-11T19:15:46.100 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:45 smithi072 ceph-mon[31695]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:15:46.157 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-03-11T19:15:46.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:45 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi143 2024-03-11T19:15:46.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:45 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 8540M 2024-03-11T19:15:46.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:45 smithi143 ceph-mon[36013]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:15:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:45 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi143 2024-03-11T19:15:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:45 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 8540M 2024-03-11T19:15:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:45 smithi106 ceph-mon[35889]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:15:46.588 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:15:46.998 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:46 smithi072 ceph-mon[31695]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:15:46.999 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:46 smithi072 ceph-mon[31695]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:46 smithi106 ceph-mon[35889]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:15:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:46 smithi106 ceph-mon[35889]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:46 smithi143 ceph-mon[36013]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:15:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:46 smithi143 ceph-mon[36013]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:47 smithi072 ceph-mon[31695]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:15:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:47 smithi106 ceph-mon[35889]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:15:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:47 smithi143 ceph-mon[36013]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:15:49.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 -- 172.21.15.72:0/2427877569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0101ca0 msgr2=0x7efee01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:49.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 --2- 172.21.15.72:0/2427877569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0101ca0 0x7efee01020c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efed00099c0 tx=0x7efed0009cd0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 -- 172.21.15.72:0/2427877569 shutdown_connections 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 --2- 172.21.15.72:0/2427877569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0103840 0x7efee010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 --2- 172.21.15.72:0/2427877569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0101ca0 0x7efee01020c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 --2- 172.21.15.72:0/2427877569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 0x7efee0103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 -- 172.21.15.72:0/2427877569 >> 172.21.15.72:0/2427877569 conn(0x7efee00fd280 msgr2=0x7efee00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 -- 172.21.15.72:0/2427877569 shutdown_connections 2024-03-11T19:15:49.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.209+0000 7efee6800700 1 -- 172.21.15.72:0/2427877569 wait complete. 2024-03-11T19:15:49.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.210+0000 7efee6800700 1 Processor -- start 2024-03-11T19:15:49.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.210+0000 7efee6800700 1 -- start start 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 0x7efee019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0103840 0x7efee01a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a1a60 con 0x7efee0103840 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a1ba0 con 0x7efee0101ca0 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efee6800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01a1d30 con 0x7efee0102ea0 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedea00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 0x7efee019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45008/0 (socket says 172.21.15.72:45008) 2024-03-11T19:15:49.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0103840 0x7efee01a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 -- 172.21.15.72:0/257810242 learned_addr learned my addr 172.21.15.72:0/257810242 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 -- 172.21.15.72:0/257810242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 msgr2=0x7efee019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 0x7efee019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 -- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0103840 msgr2=0x7efee01a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.211+0000 7efedf400700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0103840 0x7efee01a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efedf400700 1 -- 172.21.15.72:0/257810242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00095f0 con 0x7efee0101ca0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efedf400700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efed00036c0 tx=0x7efed00055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed001d070 con 0x7efee0101ca0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efed0003e90 con 0x7efee0101ca0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee01a1fb0 con 0x7efee0101ca0 2024-03-11T19:15:49.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000fc80 con 0x7efee0101ca0 2024-03-11T19:15:49.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.212+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01a2310 con 0x7efee0101ca0 2024-03-11T19:15:49.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.213+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee010d630 con 0x7efee0101ca0 2024-03-11T19:15:49.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.215+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efed000b9a0 con 0x7efee0101ca0 2024-03-11T19:15:49.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:48 smithi072 ceph-mon[31695]: osdmap e43: 6 total, 6 up, 6 in 2024-03-11T19:15:49.220 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:48 smithi072 ceph-mon[31695]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:49.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.216+0000 7efed7e00700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 0x7efec40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:15:49.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.216+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7efed00cd2b0 con 0x7efee0101ca0 2024-03-11T19:15:49.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.217+0000 7efedea00700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 0x7efec40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:15:49.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.217+0000 7efedea00700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 0x7efec40a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efecc009c60 tx=0x7efecc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:15:49.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.220+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efed0097370 con 0x7efee0101ca0 2024-03-11T19:15:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:48 smithi106 ceph-mon[35889]: osdmap e43: 6 total, 6 up, 6 in 2024-03-11T19:15:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:48 smithi106 ceph-mon[35889]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:49.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:48 smithi143 ceph-mon[36013]: osdmap e43: 6 total, 6 up, 6 in 2024-03-11T19:15:49.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:48 smithi143 ceph-mon[36013]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:49.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:15:49.452+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efee00755c0 con 0x7efec40a1ac0 2024-03-11T19:15:50.355 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:50.355 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:50.355 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:50.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:50.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:50.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:15:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:15:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:15:51.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:50 smithi106 ceph-mon[35889]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:51.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:50 smithi106 ceph-mon[35889]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:51.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:50 smithi072 ceph-mon[31695]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:51.365 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:50 smithi072 ceph-mon[31695]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:50 smithi143 ceph-mon[36013]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:15:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:50 smithi143 ceph-mon[36013]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:15:53.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:52 smithi072 ceph-mon[31695]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-11T19:15:53.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:52 smithi106 ceph-mon[35889]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-11T19:15:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:52 smithi143 ceph-mon[36013]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-11T19:15:54.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:53 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]: dispatch 2024-03-11T19:15:54.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:53 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]': finished 2024-03-11T19:15:54.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:53 smithi072 ceph-mon[31695]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:15:54.372 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:15:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:53 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]: dispatch 2024-03-11T19:15:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:53 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]': finished 2024-03-11T19:15:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:53 smithi106 ceph-mon[35889]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:15:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:15:54.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:53 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]: dispatch 2024-03-11T19:15:54.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:53 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1934894746' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b"}]': finished 2024-03-11T19:15:54.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:53 smithi143 ceph-mon[36013]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:15:54.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:15:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:54 smithi106 ceph-mon[35889]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-11T19:15:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:54 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/855668480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:55.417 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:54 smithi143 ceph-mon[36013]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-11T19:15:55.417 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:54 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/855668480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:54 smithi072 ceph-mon[31695]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-11T19:15:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:54 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/855668480' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:15:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:57 smithi106 ceph-mon[35889]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:15:57.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:57 smithi072 ceph-mon[31695]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:15:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:57 smithi143 ceph-mon[36013]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:15:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:15:59 smithi106 ceph-mon[35889]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:15:59.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:15:59 smithi072 ceph-mon[31695]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:15:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:15:59 smithi143 ceph-mon[36013]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:16:01.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:01 smithi106 ceph-mon[35889]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:16:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:01 smithi072 ceph-mon[31695]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:16:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:01 smithi143 ceph-mon[36013]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:16:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:03 smithi106 ceph-mon[35889]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:03.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:03 smithi072 ceph-mon[31695]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:03 smithi143 ceph-mon[36013]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:04 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:16:04.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:04 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:04.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:04 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:16:04.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:04 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:04 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:16:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:04 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:05.285 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:05 smithi072 ceph-mon[31695]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:05.286 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:05 smithi072 ceph-mon[31695]: Deploying daemon osd.6 on smithi072 2024-03-11T19:16:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:05 smithi106 ceph-mon[35889]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:05 smithi106 ceph-mon[35889]: Deploying daemon osd.6 on smithi072 2024-03-11T19:16:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:05 smithi143 ceph-mon[36013]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:05 smithi143 ceph-mon[36013]: Deploying daemon osd.6 on smithi072 2024-03-11T19:16:07.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:07 smithi072 ceph-mon[31695]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:07.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:07 smithi106 ceph-mon[35889]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:07 smithi143 ceph-mon[36013]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:09.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:09 smithi072 ceph-mon[31695]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:09.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:09 smithi106 ceph-mon[35889]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:09 smithi143 ceph-mon[36013]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:11.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:11 smithi072 ceph-mon[31695]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:11 smithi106 ceph-mon[35889]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:11 smithi143 ceph-mon[36013]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:13.374 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:13 smithi143 ceph-mon[36013]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:13.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:13 smithi106 ceph-mon[35889]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:13.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:13 smithi072 ceph-mon[31695]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:14 smithi143 ceph-mon[36013]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:14.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:14.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:14 smithi072 ceph-mon[31695]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:14 smithi106 ceph-mon[35889]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:17.367 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:17 smithi072 ceph-mon[31695]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:17.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:17 smithi106 ceph-mon[35889]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:17 smithi143 ceph-mon[36013]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:18.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:18 smithi072 ceph-mon[31695]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:16:18.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:18.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:18 smithi106 ceph-mon[35889]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:16:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:18 smithi143 ceph-mon[36013]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:16:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:19.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.210+0000 7efed7e00700 1 -- 172.21.15.72:0/257810242 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efee00755c0 con 0x7efec40a1ac0 2024-03-11T19:16:19.214 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 6 on host 'smithi072' 2024-03-11T19:16:19.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.213+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 msgr2=0x7efec40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:19.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.213+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 0x7efec40a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efecc009c60 tx=0x7efecc009260 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.213+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 msgr2=0x7efee0199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.213+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efed00036c0 tx=0x7efed00055f0 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 shutdown_connections 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efec40a1ac0 0x7efec40a3f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efee0103840 0x7efee01a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efee0101ca0 0x7efee0199f70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 --2- 172.21.15.72:0/257810242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee0102ea0 0x7efee019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 >> 172.21.15.72:0/257810242 conn(0x7efee00fd280 msgr2=0x7efee0073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 shutdown_connections 2024-03-11T19:16:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:16:19.214+0000 7efee6800700 1 -- 172.21.15.72:0/257810242 wait complete. 2024-03-11T19:16:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:19 smithi106 ceph-mon[35889]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:19 smithi106 ceph-mon[35889]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:16:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:19 smithi106 ceph-mon[35889]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:16:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:19 smithi106 ceph-mon[35889]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:16:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:19.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:19 smithi072 ceph-mon[31695]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:19.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:19 smithi072 ceph-mon[31695]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:16:19.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:19 smithi072 ceph-mon[31695]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:16:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:19 smithi072 ceph-mon[31695]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:16:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:19 smithi143 ceph-mon[36013]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:19 smithi143 ceph-mon[36013]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:16:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:19 smithi143 ceph-mon[36013]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:16:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:19 smithi143 ceph-mon[36013]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:16:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.009 DEBUG:teuthology.orchestra.run.smithi072:osd.6> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.6.service 2024-03-11T19:16:20.013 INFO:tasks.cephadm:Deploying osd.7 on smithi106 with /dev/vg_nvme/lv_2... 2024-03-11T19:16:20.013 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:16:20.246 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:16:20.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:16:20.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: osdmap e46: 7 total, 6 up, 7 in 2024-03-11T19:16:20.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:20.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:20 smithi072 ceph-mon[31695]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' 2024-03-11T19:16:20.367 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:16:20.367 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: osdmap e46: 7 total, 6 up, 7 in 2024-03-11T19:16:20.367 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:20.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:20 smithi106 ceph-mon[35889]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' 2024-03-11T19:16:20.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:16:20.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: osdmap e46: 7 total, 6 up, 7 in 2024-03-11T19:16:20.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:20.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:20.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:20 smithi143 ceph-mon[36013]: from='osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062]' entity='osd.6' 2024-03-11T19:16:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:16:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:16:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:21.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] boot 2024-03-11T19:16:21.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:16:21.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] boot 2024-03-11T19:16:21.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:16:21.328 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:16:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:16:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:16:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:21.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: osd.6 [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] boot 2024-03-11T19:16:21.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:16:21.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:16:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:16:22.205 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:16:22.205 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-11T19:16:22.206 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-11T19:16:22.206 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120167 s, 873 MB/s 2024-03-11T19:16:22.206 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-11T19:16:22.811 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-03-11T19:16:23.052 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:16:23.186 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:23 smithi072 ceph-mon[31695]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:23.186 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:23 smithi072 ceph-mon[31695]: osdmap e48: 7 total, 7 up, 7 in 2024-03-11T19:16:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:23 smithi106 ceph-mon[35889]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:23 smithi106 ceph-mon[35889]: osdmap e48: 7 total, 7 up, 7 in 2024-03-11T19:16:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:23 smithi143 ceph-mon[36013]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:23 smithi143 ceph-mon[36013]: osdmap e48: 7 total, 7 up, 7 in 2024-03-11T19:16:24.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 -- 172.21.15.106:0/4268413139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 msgr2=0x7fdef410a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:24.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 --2- 172.21.15.106:0/4268413139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef410a110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00b0d0 tx=0x7fdeec00b3e0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 -- 172.21.15.106:0/4268413139 shutdown_connections 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 --2- 172.21.15.106:0/4268413139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef410a110 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 --2- 172.21.15.106:0/4268413139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 --2- 172.21.15.106:0/4268413139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdef4102ec0 0x7fdef4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 -- 172.21.15.106:0/4268413139 >> 172.21.15.106:0/4268413139 conn(0x7fdef40fd280 msgr2=0x7fdef40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 -- 172.21.15.106:0/4268413139 shutdown_connections 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.871+0000 7fdef968a700 1 -- 172.21.15.106:0/4268413139 wait complete. 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 Processor -- start 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 -- start start 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdef4102ec0 0x7fdef41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef419f8f0 con 0x7fdef4102ec0 2024-03-11T19:16:24.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef419fa30 con 0x7fdef4103860 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef968a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef419fbf0 con 0x7fdef4101cc0 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51238/0 (socket says 172.21.15.106:51238) 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 -- 172.21.15.106:0/4008000981 learned_addr learned my addr 172.21.15.106:0/4008000981 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef2ffd700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 -- 172.21.15.106:0/4008000981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 msgr2=0x7fdef4197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef4197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 -- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdef4102ec0 msgr2=0x7fdef41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdef4102ec0 0x7fdef41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 -- 172.21.15.106:0/4008000981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec009d00 con 0x7fdef4103860 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef2ffd700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:16:24.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef37fe700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007c90 tx=0x7fdeec009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.872+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00e040 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.873+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdeec004e60 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.873+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef419fe70 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.873+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec012bc0 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.873+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef41a02a0 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.874+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4109020 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.874+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdeec012d20 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.875+0000 7fdef0ff9700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 0x7fdee00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.875+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fdeec0cdd50 con 0x7fdef4103860 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.875+0000 7fdef2ffd700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 0x7fdee00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.876+0000 7fdef2ffd700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 0x7fdee00a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005fd0 tx=0x7fdee4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:16:24.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:24.880+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdeec0d1030 con 0x7fdef4103860 2024-03-11T19:16:25.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:25.081+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdef402d910 con 0x7fdee00a1ac0 2024-03-11T19:16:25.144 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:25 smithi106 ceph-mon[35889]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:25 smithi072 ceph-mon[31695]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:25 smithi143 ceph-mon[36013]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:26.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:26.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:26.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:26.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:26.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:26.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:27 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi072 2024-03-11T19:16:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:27 smithi106 ceph-mon[35889]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:27 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 5696M 2024-03-11T19:16:27.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:27 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi072 2024-03-11T19:16:27.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:27 smithi072 ceph-mon[31695]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:27.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:27 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 5696M 2024-03-11T19:16:27.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:27 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi072 2024-03-11T19:16:27.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:27 smithi143 ceph-mon[36013]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:27.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:27 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 5696M 2024-03-11T19:16:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:28 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/2829066651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:28 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:28 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]': finished 2024-03-11T19:16:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:28 smithi106 ceph-mon[35889]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:16:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:28 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:28.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:28 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/2829066651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:28 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:28 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]': finished 2024-03-11T19:16:28.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:28 smithi072 ceph-mon[31695]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:16:28.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:28 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:28 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/2829066651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:28 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]: dispatch 2024-03-11T19:16:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:28 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8358cb46-e68c-49bd-9add-390afbd8ef8e"}]': finished 2024-03-11T19:16:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:28 smithi143 ceph-mon[36013]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:16:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:28 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:29.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:29 smithi106 ceph-mon[35889]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:29.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:29 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/3021548141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:29.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:29 smithi072 ceph-mon[31695]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:29.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:29 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/3021548141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:29.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:29 smithi143 ceph-mon[36013]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:29.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:29 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/3021548141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:31 smithi106 ceph-mon[35889]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:31 smithi072 ceph-mon[31695]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:31.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:31 smithi143 ceph-mon[36013]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:33 smithi106 ceph-mon[35889]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:33.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:33 smithi072 ceph-mon[31695]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:33 smithi143 ceph-mon[36013]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:35 smithi106 ceph-mon[35889]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:35.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:35 smithi072 ceph-mon[31695]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:35 smithi143 ceph-mon[36013]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:16:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:36 smithi106 ceph-mon[35889]: Deploying daemon osd.7 on smithi106 2024-03-11T19:16:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:16:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:36 smithi072 ceph-mon[31695]: Deploying daemon osd.7 on smithi106 2024-03-11T19:16:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:16:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:36 smithi143 ceph-mon[36013]: Deploying daemon osd.7 on smithi106 2024-03-11T19:16:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:37 smithi106 ceph-mon[35889]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:37.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:37 smithi072 ceph-mon[31695]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:37 smithi143 ceph-mon[36013]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:39.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:39 smithi106 ceph-mon[35889]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:39.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:39 smithi072 ceph-mon[31695]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:39 smithi143 ceph-mon[36013]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:41 smithi072 ceph-mon[31695]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:41 smithi143 ceph-mon[36013]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:41 smithi106 ceph-mon[35889]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:43.363 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:43 smithi106 ceph-mon[35889]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:43.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:43 smithi072 ceph-mon[31695]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:43 smithi143 ceph-mon[36013]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:44.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.788+0000 7fdef0ff9700 1 -- 172.21.15.106:0/4008000981 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdef402d910 con 0x7fdee00a1ac0 2024-03-11T19:16:44.791 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 7 on host 'smithi106' 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 msgr2=0x7fdee00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 0x7fdee00a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005fd0 tx=0x7fdee4005dd0 comp rx=0 tx=0).stop 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 msgr2=0x7fdef419f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007c90 tx=0x7fdeec009430 comp rx=0 tx=0).stop 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 shutdown_connections 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdee00a1ac0 0x7fdee00a3f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdef4103860 0x7fdef419f320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdef4101cc0 0x7fdef4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:44.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 --2- 172.21.15.106:0/4008000981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdef4102ec0 0x7fdef41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:44.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 >> 172.21.15.106:0/4008000981 conn(0x7fdef40fd280 msgr2=0x7fdef4106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:16:44.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 shutdown_connections 2024-03-11T19:16:44.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:16:44.791+0000 7fdef968a700 1 -- 172.21.15.106:0/4008000981 wait complete. 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:45 smithi106 ceph-mon[35889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:45 smithi072 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.433 DEBUG:teuthology.orchestra.run.smithi106:osd.7> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.7.service 2024-03-11T19:16:45.435 INFO:tasks.cephadm:Deploying osd.8 on smithi143 with /dev/vg_nvme/lv_2... 2024-03-11T19:16:45.435 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:45 smithi143 ceph-mon[36013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:16:45.680 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:16:47.142 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:16:47.142 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.142 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:16:47.142 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:47.142 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.143 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:46 smithi143 ceph-mon[36013]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:46 smithi106 ceph-mon[35889]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: from='osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:16:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:46 smithi072 ceph-mon[31695]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:16:47.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:16:47.316+0000 7f685d9c3700 -1 osd.7 0 waiting for initial osdmap 2024-03-11T19:16:47.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:16:47.326+0000 7f6854fad700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123311 s, 850 MB/s 2024-03-11T19:16:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: osdmap e51: 8 total, 7 up, 8 in 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: from='osd.7 ' entity='osd.7' 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] boot 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:16:48.038 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.096 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: osdmap e51: 8 total, 7 up, 8 in 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: from='osd.7 ' entity='osd.7' 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] boot 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:16:48.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: osdmap e51: 8 total, 7 up, 8 in 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: from='osd.7 ' entity='osd.7' 2024-03-11T19:16:48.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: osd.7 [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] boot 2024-03-11T19:16:48.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:16:48.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:16:48.358 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-03-11T19:16:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:48 smithi106 ceph-mon[35889]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:16:49.151 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:49.151 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.151 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.151 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.151 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.152 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:48 smithi143 ceph-mon[36013]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:16:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:16:49.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:48 smithi072 ceph-mon[31695]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:16:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:49 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi106 2024-03-11T19:16:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:49 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-11T19:16:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:49 smithi106 ceph-mon[35889]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:16:50.168 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:49 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi106 2024-03-11T19:16:50.169 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:49 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-11T19:16:50.169 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:49 smithi143 ceph-mon[36013]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:16:50.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:49 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi106 2024-03-11T19:16:50.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:49 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-11T19:16:50.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:49 smithi072 ceph-mon[31695]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:16:50.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/1578337058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 msgr2=0x7fdd28109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:50.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/1578337058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd28109ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b0a0 tx=0x7fdd2000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/1578337058 shutdown_connections 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/1578337058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd28109ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/1578337058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd281027d0 0x7fdd28104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/1578337058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd28105100 0x7fdd281074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/1578337058 >> 172.21.15.143:0/1578337058 conn(0x7fdd280fc200 msgr2=0x7fdd280fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/1578337058 shutdown_connections 2024-03-11T19:16:50.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.401+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/1578337058 wait complete. 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 Processor -- start 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 -- start start 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd281027d0 0x7fdd28197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28105100 0x7fdd28198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819f8e0 con 0x7fdd28105100 2024-03-11T19:16:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819fa20 con 0x7fdd281027d0 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd2cb8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819fbb0 con 0x7fdd28107a30 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd26d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd26d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53818/0 (socket says 172.21.15.143:53818) 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.402+0000 7fdd26d9d700 1 -- 172.21.15.143:0/3717810038 learned_addr learned my addr 172.21.15.143:0/3717810038 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 -- 172.21.15.143:0/3717810038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd281027d0 msgr2=0x7fdd28197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd281027d0 0x7fdd28197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 -- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28105100 msgr2=0x7fdd28198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28105100 0x7fdd28198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 -- 172.21.15.143:0/3717810038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009d00 con 0x7fdd28107a30 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd26d9d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd200094f0 tx=0x7fdd200095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000e040 con 0x7fdd28107a30 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2819fe30 con 0x7fdd28107a30 2024-03-11T19:16:50.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd2001db60 con 0x7fdd28107a30 2024-03-11T19:16:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2001bbd0 con 0x7fdd28107a30 2024-03-11T19:16:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.403+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd281a0270 con 0x7fdd28107a30 2024-03-11T19:16:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.404+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd280fdd80 con 0x7fdd28107a30 2024-03-11T19:16:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.405+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdd2001bd30 con 0x7fdd28107a30 2024-03-11T19:16:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.406+0000 7fdd177fe700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 0x7fdd100a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:16:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.406+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fdd200cec20 con 0x7fdd28107a30 2024-03-11T19:16:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.406+0000 7fdd2659c700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 0x7fdd100a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:16:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.407+0000 7fdd2659c700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 0x7fdd100a3f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd180097b0 tx=0x7fdd18006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:16:50.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.409+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd20098b20 con 0x7fdd28107a30 2024-03-11T19:16:50.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:16:50.615+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd2802da00 con 0x7fdd100a1ac0 2024-03-11T19:16:51.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:50 smithi072 ceph-mon[31695]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:51.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:51.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:51.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:50 smithi143 ceph-mon[36013]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:51.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:50 smithi106 ceph-mon[35889]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:51.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:16:51.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:16:51.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:16:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:51 smithi072 ceph-mon[31695]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:51 smithi072 ceph-mon[31695]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:16:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:51 smithi072 ceph-mon[31695]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-11T19:16:52.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:51 smithi072 ceph-mon[31695]: osdmap e56: 8 total, 8 up, 8 in 2024-03-11T19:16:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:51 smithi143 ceph-mon[36013]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:51 smithi143 ceph-mon[36013]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:16:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:51 smithi143 ceph-mon[36013]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-11T19:16:52.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:51 smithi143 ceph-mon[36013]: osdmap e56: 8 total, 8 up, 8 in 2024-03-11T19:16:52.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:51 smithi106 ceph-mon[35889]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:16:52.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:51 smithi106 ceph-mon[35889]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:16:52.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:51 smithi106 ceph-mon[35889]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-11T19:16:52.240 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:51 smithi106 ceph-mon[35889]: osdmap e56: 8 total, 8 up, 8 in 2024-03-11T19:16:53.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:52 smithi072 ceph-mon[31695]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:53.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:52 smithi143 ceph-mon[36013]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:53.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:52 smithi106 ceph-mon[35889]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:53 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/3116072630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:53 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:53 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]': finished 2024-03-11T19:16:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:53 smithi072 ceph-mon[31695]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:16:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:16:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:53 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/3116072630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:53 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:53 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]': finished 2024-03-11T19:16:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:53 smithi143 ceph-mon[36013]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:16:54.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:16:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:53 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/3116072630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:53 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]: dispatch 2024-03-11T19:16:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:53 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c936701-2ea3-476a-abad-ecaaf2bdc9e4"}]': finished 2024-03-11T19:16:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:53 smithi106 ceph-mon[35889]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:16:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:16:55.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:54 smithi143 ceph-mon[36013]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:55.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:54 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/3693889037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:54 smithi106 ceph-mon[35889]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:54 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/3693889037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:54 smithi072 ceph-mon[31695]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:54 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/3693889037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:16:56.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:55 smithi143 ceph-mon[36013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-11T19:16:56.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:55 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:16:56.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:55 smithi106 ceph-mon[35889]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-11T19:16:56.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:55 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:16:56.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:55 smithi072 ceph-mon[31695]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-11T19:16:56.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:55 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:16:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:56 smithi106 ceph-mon[35889]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:57.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:56 smithi072 ceph-mon[31695]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:56 smithi143 ceph-mon[36013]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:59.060 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:16:59 smithi143 ceph-mon[36013]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:16:59 smithi106 ceph-mon[35889]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:16:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:16:59 smithi072 ceph-mon[31695]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:01.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:01 smithi106 ceph-mon[35889]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:01.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:01 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:17:01.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:01 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:01 smithi072 ceph-mon[31695]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:01 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:17:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:01 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:01 smithi143 ceph-mon[36013]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:01 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:17:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:01 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:02.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:02 smithi106 ceph-mon[35889]: Deploying daemon osd.8 on smithi143 2024-03-11T19:17:02.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:02 smithi072 ceph-mon[31695]: Deploying daemon osd.8 on smithi143 2024-03-11T19:17:02.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:02 smithi143 ceph-mon[36013]: Deploying daemon osd.8 on smithi143 2024-03-11T19:17:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:03 smithi106 ceph-mon[35889]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:03.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:03 smithi072 ceph-mon[31695]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:03 smithi143 ceph-mon[36013]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:05.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:05 smithi143 ceph-mon[36013]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:05 smithi106 ceph-mon[35889]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:05 smithi072 ceph-mon[31695]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:07.317 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:07 smithi143 ceph-mon[36013]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:07.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.318 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:07 smithi106 ceph-mon[35889]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:07 smithi072 ceph-mon[31695]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:09.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:09 smithi106 ceph-mon[35889]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:09.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:09 smithi072 ceph-mon[31695]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:09 smithi143 ceph-mon[36013]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:10.457 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 8 on host 'smithi143' 2024-03-11T19:17:10.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.453+0000 7fdd177fe700 1 -- 172.21.15.143:0/3717810038 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdd2802da00 con 0x7fdd100a1ac0 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 msgr2=0x7fdd100a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 0x7fdd100a3f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd180097b0 tx=0x7fdd18006d20 comp rx=0 tx=0).stop 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 msgr2=0x7fdd2819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd200094f0 tx=0x7fdd200095d0 comp rx=0 tx=0).stop 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 shutdown_connections 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd100a1ac0 0x7fdd100a3f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:10.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd28107a30 0x7fdd2819f310 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:10.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd281027d0 0x7fdd28197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:10.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 --2- 172.21.15.143:0/3717810038 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd28105100 0x7fdd28198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:10.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 >> 172.21.15.143:0/3717810038 conn(0x7fdd280fc200 msgr2=0x7fdd281030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:17:10.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 shutdown_connections 2024-03-11T19:17:10.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:17:10.456+0000 7fdd2cb8d700 1 -- 172.21.15.143:0/3717810038 wait complete. 2024-03-11T19:17:11.164 DEBUG:teuthology.orchestra.run.smithi143:osd.8> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.8.service 2024-03-11T19:17:11.167 INFO:tasks.cephadm:Deploying osd.9 on smithi072 with /dev/vg_nvme/lv_1... 2024-03-11T19:17:11.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:17:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:11 smithi106 ceph-mon[35889]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:11 smithi106 ceph-mon[35889]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:11 smithi106 ceph-mon[35889]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:11 smithi072 ceph-mon[31695]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:11 smithi072 ceph-mon[31695]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:11 smithi072 ceph-mon[31695]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:11 smithi143 ceph-mon[36013]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:11 smithi143 ceph-mon[36013]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:11 smithi143 ceph-mon[36013]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:17:11.613 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:17:12.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:17:12.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:17:12.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:12.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.358 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:12 smithi072 ceph-mon[31695]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.428 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-mon[36013]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:17:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:17:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:12 smithi106 ceph-mon[35889]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:12.689 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:17:12.423+0000 7eff2c36f700 -1 osd.8 0 waiting for initial osdmap 2024-03-11T19:17:12.689 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:17:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:17:12.433+0000 7eff28162700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:17:13.493 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:13 smithi072 ceph-mon[31695]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:17:13.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:13 smithi072 ceph-mon[31695]: osdmap e59: 9 total, 8 up, 9 in 2024-03-11T19:17:13.494 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:13 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:13.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:13 smithi143 ceph-mon[36013]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:17:13.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:13 smithi143 ceph-mon[36013]: osdmap e59: 9 total, 8 up, 9 in 2024-03-11T19:17:13.555 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:13 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:13.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:13 smithi106 ceph-mon[35889]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:17:13.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:13 smithi106 ceph-mon[35889]: osdmap e59: 9 total, 8 up, 9 in 2024-03-11T19:17:13.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:13 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] boot 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.581 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:14 smithi143 ceph-mon[36013]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] boot 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:17:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:14 smithi072 ceph-mon[31695]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: osd.8 [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] boot 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:17:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:14 smithi106 ceph-mon[35889]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0348657 s, 301 MB/s 2024-03-11T19:17:14.968 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-03-11T19:17:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:17:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi143 2024-03-11T19:17:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 5693M 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi143 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 5693M 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi143 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 5693M 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:16.112 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi072:vg_nvme/lv_1 2024-03-11T19:17:16.524 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:17:16.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:17:16.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:17:16.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:16.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:16.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:16.753 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:17:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:17:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:18.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:17 smithi072 ceph-mon[31695]: osdmap e64: 9 total, 9 up, 9 in 2024-03-11T19:17:18.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:17 smithi143 ceph-mon[36013]: osdmap e64: 9 total, 9 up, 9 in 2024-03-11T19:17:18.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:17 smithi106 ceph-mon[35889]: osdmap e64: 9 total, 9 up, 9 in 2024-03-11T19:17:19.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 -- 172.21.15.72:0/3763795423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c075c20 msgr2=0x7f1b3c074080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 --2- 172.21.15.72:0/3763795423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c075c20 0x7f1b3c074080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009990 tx=0x7f1b2c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 -- 172.21.15.72:0/3763795423 shutdown_connections 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 --2- 172.21.15.72:0/3763795423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c109bf0 0x7f1b3c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 --2- 172.21.15.72:0/3763795423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c075c20 0x7f1b3c074080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 --2- 172.21.15.72:0/3763795423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c074650 0x7f1b3c074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 -- 172.21.15.72:0/3763795423 >> 172.21.15.72:0/3763795423 conn(0x7f1b3c0fd290 msgr2=0x7f1b3c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:17:19.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 -- 172.21.15.72:0/3763795423 shutdown_connections 2024-03-11T19:17:19.141 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:18 smithi072 ceph-mon[31695]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.135+0000 7f1b41c00700 1 -- 172.21.15.72:0/3763795423 wait complete. 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.136+0000 7f1b41c00700 1 Processor -- start 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.136+0000 7f1b41c00700 1 -- start start 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c075c20 0x7f1b3c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:19.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c109bf0 0x7f1b3c1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c1a1980 con 0x7f1b3c074650 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c1a1ac0 con 0x7f1b3c109bf0 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b41c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c1a1c50 con 0x7f1b3c075c20 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46470/0 (socket says 172.21.15.72:46470) 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 -- 172.21.15.72:0/4258179018 learned_addr learned my addr 172.21.15.72:0/4258179018 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3b400700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c109bf0 0x7f1b3c1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3a000700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c075c20 0x7f1b3c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 -- 172.21.15.72:0/4258179018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c075c20 msgr2=0x7f1b3c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:19.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c075c20 0x7f1b3c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 -- 172.21.15.72:0/4258179018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c109bf0 msgr2=0x7f1b3c1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c109bf0 0x7f1b3c1a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.137+0000 7f1b3aa00700 1 -- 172.21.15.72:0/4258179018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28009520 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b3aa00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b650 tx=0x7f1b2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c01d070 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b2c005030 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c0095f0 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c004ce0 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.138+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c1a2150 con 0x7f1b3c074650 2024-03-11T19:17:19.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.139+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c108a00 con 0x7f1b3c074650 2024-03-11T19:17:19.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.141+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1b2c005510 con 0x7f1b3c074650 2024-03-11T19:17:19.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.143+0000 7f1b33400700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 0x7f1b200a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:19.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.144+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f1b2c0cdd10 con 0x7f1b3c074650 2024-03-11T19:17:19.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.144+0000 7f1b3a000700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 0x7f1b200a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:19.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.144+0000 7f1b3a000700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 0x7f1b200a4000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009df0 tx=0x7f1b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:17:19.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.147+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b2c0978c0 con 0x7f1b3c074650 2024-03-11T19:17:19.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:18 smithi143 ceph-mon[36013]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:17:19.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:19.378+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b3c02d910 con 0x7f1b200a1b40 2024-03-11T19:17:19.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:18 smithi106 ceph-mon[35889]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:17:20.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:20.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:20.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:20.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:20.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:20.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:20 smithi072 ceph-mon[31695]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:21.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:20 smithi072 ceph-mon[31695]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-11T19:17:21.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:20 smithi143 ceph-mon[36013]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:21.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:20 smithi143 ceph-mon[36013]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-11T19:17:21.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:20 smithi106 ceph-mon[35889]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:21.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:20 smithi106 ceph-mon[35889]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-11T19:17:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:23 smithi106 ceph-mon[35889]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-11T19:17:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:23 smithi072 ceph-mon[31695]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-11T19:17:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:23 smithi143 ceph-mon[36013]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-11T19:17:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:24 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]: dispatch 2024-03-11T19:17:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:24 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]': finished 2024-03-11T19:17:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:24 smithi106 ceph-mon[35889]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:17:24.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:24 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:24 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]: dispatch 2024-03-11T19:17:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:24 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]': finished 2024-03-11T19:17:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:24 smithi072 ceph-mon[31695]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:17:24.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:24 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:24.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:24 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]: dispatch 2024-03-11T19:17:24.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:24 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/9936288' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7124f938-a535-4c94-8959-2931abf1fdd0"}]': finished 2024-03-11T19:17:24.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:24 smithi143 ceph-mon[36013]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:17:24.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:24 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:25.332 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:25 smithi072 ceph-mon[31695]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:17:25.332 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:25 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2946100860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:25 smithi106 ceph-mon[35889]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:17:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:25 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2946100860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:25 smithi143 ceph-mon[36013]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:17:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:25 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2946100860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:27.337 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:27 smithi072 ceph-mon[31695]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:17:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:27 smithi106 ceph-mon[35889]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:17:27.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:27 smithi143 ceph-mon[36013]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:17:29.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:29 smithi106 ceph-mon[35889]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:29.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:29 smithi072 ceph-mon[31695]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:29.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:29 smithi143 ceph-mon[36013]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:31.390 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:31 smithi072 ceph-mon[31695]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:31 smithi106 ceph-mon[35889]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:31.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:31 smithi143 ceph-mon[36013]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:33.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:33 smithi072 ceph-mon[31695]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:33 smithi106 ceph-mon[35889]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:33 smithi143 ceph-mon[36013]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:34.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:17:34.347 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:34.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:17:34.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:17:34.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:35 smithi106 ceph-mon[35889]: Deploying daemon osd.9 on smithi072 2024-03-11T19:17:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:35 smithi106 ceph-mon[35889]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:35.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:35 smithi072 ceph-mon[31695]: Deploying daemon osd.9 on smithi072 2024-03-11T19:17:35.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:35 smithi072 ceph-mon[31695]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:35 smithi143 ceph-mon[36013]: Deploying daemon osd.9 on smithi072 2024-03-11T19:17:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:35 smithi143 ceph-mon[36013]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:37.171 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:37 smithi072 ceph-mon[31695]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:37 smithi106 ceph-mon[35889]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:37 smithi143 ceph-mon[36013]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:39.305 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:39 smithi072 ceph-mon[31695]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:39.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:39 smithi106 ceph-mon[35889]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:39 smithi143 ceph-mon[36013]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:41 smithi072 ceph-mon[31695]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:41 smithi106 ceph-mon[35889]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:41 smithi143 ceph-mon[36013]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:43 smithi106 ceph-mon[35889]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:43.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:43 smithi072 ceph-mon[31695]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:43 smithi143 ceph-mon[36013]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:45 smithi106 ceph-mon[35889]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:45 smithi072 ceph-mon[31695]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:45 smithi143 ceph-mon[36013]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:47.280 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:47 smithi072 ceph-mon[31695]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:47.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:47 smithi072 ceph-mon[31695]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:17:47.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:47.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:47 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:47 smithi106 ceph-mon[35889]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:47 smithi106 ceph-mon[35889]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:17:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:47 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:47 smithi143 ceph-mon[36013]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:47 smithi143 ceph-mon[36013]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:17:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:47 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:48.238 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 9 on host 'smithi072' 2024-03-11T19:17:48.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.234+0000 7f1b33400700 1 -- 172.21.15.72:0/4258179018 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b3c02d910 con 0x7f1b200a1b40 2024-03-11T19:17:48.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 msgr2=0x7f1b200a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:48.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 0x7f1b200a4000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009df0 tx=0x7f1b28009260 comp rx=0 tx=0).stop 2024-03-11T19:17:48.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 msgr2=0x7f1b3c199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b650 tx=0x7f1b2c004730 comp rx=0 tx=0).stop 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 shutdown_connections 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1b200a1b40 0x7f1b200a4000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b3c109bf0 0x7f1b3c1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b3c074650 0x7f1b3c199df0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 --2- 172.21.15.72:0/4258179018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b3c075c20 0x7f1b3c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.239+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 >> 172.21.15.72:0/4258179018 conn(0x7f1b3c0fd290 msgr2=0x7f1b3c105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:17:48.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.240+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 shutdown_connections 2024-03-11T19:17:48.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:17:48.240+0000 7f1b41c00700 1 -- 172.21.15.72:0/4258179018 wait complete. 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: osdmap e67: 10 total, 9 up, 10 in 2024-03-11T19:17:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:48 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:17:48.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:17:48.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:17:48.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:48.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:17:48.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: osdmap e67: 10 total, 9 up, 10 in 2024-03-11T19:17:48.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:48 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: from='osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: osdmap e67: 10 total, 9 up, 10 in 2024-03-11T19:17:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:48 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:49.239 DEBUG:teuthology.orchestra.run.smithi072:osd.9> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.9.service 2024-03-11T19:17:49.246 INFO:tasks.cephadm:Deploying osd.10 on smithi106 with /dev/vg_nvme/lv_1... 2024-03-11T19:17:49.246 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:17:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:49 smithi072 ceph-mon[31695]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:49.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:49 smithi143 ceph-mon[36013]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:49.491 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:17:49.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:49 smithi106 ceph-mon[35889]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:49.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:49.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:49.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] boot 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:17:50.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:50 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] boot 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:17:50.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:50 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:50.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:17:50.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:17:50.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:50.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: osd.9 [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] boot 2024-03-11T19:17:50.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:17:50.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:50 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:17:51.248 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:51 smithi106 ceph-mon[35889]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:51.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:51 smithi106 ceph-mon[35889]: osdmap e69: 10 total, 10 up, 10 in 2024-03-11T19:17:51.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:51 smithi072 ceph-mon[31695]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:51.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:51 smithi072 ceph-mon[31695]: osdmap e69: 10 total, 10 up, 10 in 2024-03-11T19:17:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:51 smithi143 ceph-mon[36013]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:17:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:51 smithi143 ceph-mon[36013]: osdmap e69: 10 total, 10 up, 10 in 2024-03-11T19:17:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:17:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:17:51.581 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-11T19:17:51.581 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-11T19:17:51.582 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012136 s, 864 MB/s 2024-03-11T19:17:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-11T19:17:52.354 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-03-11T19:17:52.587 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:17:53.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:53 smithi106 ceph-mon[35889]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:53 smithi072 ceph-mon[31695]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:53 smithi143 ceph-mon[36013]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1774093510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 msgr2=0x7f1fdc1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1774093510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc1020c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80099c0 tx=0x7f1fd8009cd0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1774093510 shutdown_connections 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1774093510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc103840 0x7f1fdc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1774093510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc1020c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1774093510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fdc102ea0 0x7f1fdc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1774093510 >> 172.21.15.106:0/1774093510 conn(0x7f1fdc0fd280 msgr2=0x7f1fdc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:17:54.293 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1774093510 shutdown_connections 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1774093510 wait complete. 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 Processor -- start 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- start start 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc102ea0 0x7f1fdc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fdc103840 0x7f1fdc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc19f880 con 0x7f1fdc103840 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc19f9c0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.287+0000 7f1fe42fd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc19fb80 con 0x7f1fdc102ea0 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36358/0 (socket says 172.21.15.106:36358) 2024-03-11T19:17:54.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 -- 172.21.15.106:0/1013745612 learned_addr learned my addr 172.21.15.106:0/1013745612 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe1898700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc102ea0 0x7f1fdc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 -- 172.21.15.106:0/1013745612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc102ea0 msgr2=0x7f1fdc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc102ea0 0x7f1fdc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 -- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fdc103840 msgr2=0x7f1fdc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fdc103840 0x7f1fdc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 -- 172.21.15.106:0/1013745612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80095f0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe2099700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009f50 tx=0x7f1fd8003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd801d070 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8022470 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc19fe00 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800f6c0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.288+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc1a02e0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.289+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc109080 con 0x7f1fdc101ca0 2024-03-11T19:17:54.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.290+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1fd80225e0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.291+0000 7f1fd37fe700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 0x7f1fc80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:17:54.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.291+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1fd80ce770 con 0x7f1fdc101ca0 2024-03-11T19:17:54.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.291+0000 7f1fe1898700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 0x7f1fc80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:17:54.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.292+0000 7f1fe1898700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 0x7f1fc80a4000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:17:54.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.294+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fd80982a0 con 0x7f1fdc101ca0 2024-03-11T19:17:54.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:17:54.495+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fdc02d870 con 0x7f1fc80a1b40 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:17:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:17:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:17:55.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:17:55.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:17:55.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:17:56.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:56 smithi072 ceph-mon[31695]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:56.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:56 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi072 2024-03-11T19:17:56.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:56 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 4272M 2024-03-11T19:17:56.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:56 smithi143 ceph-mon[36013]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:56.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:56 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi072 2024-03-11T19:17:56.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:56 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 4272M 2024-03-11T19:17:56.519 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:56 smithi106 ceph-mon[35889]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:17:56.519 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:56 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi072 2024-03-11T19:17:56.519 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:56 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 4272M 2024-03-11T19:17:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:57 smithi143 ceph-mon[36013]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:57 smithi106 ceph-mon[35889]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:57.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:57 smithi072 ceph-mon[31695]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:58.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/3061688196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]': finished 2024-03-11T19:17:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:17:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:17:58.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:58 smithi143 ceph-mon[36013]: from='client.? 172.21.15.106:0/221519018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/3061688196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]': finished 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:17:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:58 smithi106 ceph-mon[35889]: from='client.? 172.21.15.106:0/221519018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/3061688196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]: dispatch 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3cca719-676d-4904-b3be-3f42aef54ae0"}]': finished 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:17:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:58 smithi072 ceph-mon[31695]: from='client.? 172.21.15.106:0/221519018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:17:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:17:59 smithi143 ceph-mon[36013]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:17:59 smithi106 ceph-mon[35889]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:17:59.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:17:59 smithi072 ceph-mon[31695]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:01.331 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:01 smithi106 ceph-mon[35889]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:01 smithi072 ceph-mon[31695]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:01.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:01 smithi143 ceph-mon[36013]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:03.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:03 smithi106 ceph-mon[35889]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:03.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:03 smithi072 ceph-mon[31695]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:03.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:03 smithi143 ceph-mon[36013]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:05.504 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:05 smithi106 ceph-mon[35889]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:05.504 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:05 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:18:05.504 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:05 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:05.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:05 smithi072 ceph-mon[31695]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:05.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:05 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:18:05.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:05 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:05 smithi143 ceph-mon[36013]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:05 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:18:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:05 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:06.479 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:06 smithi106 ceph-mon[35889]: Deploying daemon osd.10 on smithi106 2024-03-11T19:18:06.503 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:06 smithi072 ceph-mon[31695]: Deploying daemon osd.10 on smithi106 2024-03-11T19:18:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:06 smithi143 ceph-mon[36013]: Deploying daemon osd.10 on smithi106 2024-03-11T19:18:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:07 smithi106 ceph-mon[35889]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:07.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:07 smithi072 ceph-mon[31695]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:07.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:07 smithi143 ceph-mon[36013]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:09.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:09 smithi106 ceph-mon[35889]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:09.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:09 smithi072 ceph-mon[31695]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:09.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:09 smithi143 ceph-mon[36013]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:11.440 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:11 smithi106 ceph-mon[35889]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:11.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:11 smithi072 ceph-mon[31695]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:11 smithi143 ceph-mon[36013]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:13.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:13 smithi106 ceph-mon[35889]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:13.491 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:13 smithi143 ceph-mon[36013]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:13 smithi072 ceph-mon[31695]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:14.403 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 10 on host 'smithi106' 2024-03-11T19:18:14.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.278+0000 7f1fd37fe700 1 -- 172.21.15.106:0/1013745612 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1fdc02d870 con 0x7f1fc80a1b40 2024-03-11T19:18:14.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 msgr2=0x7f1fc80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 0x7f1fc80a4000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc008040 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 msgr2=0x7f1fdc197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009f50 tx=0x7f1fd8003960 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 shutdown_connections 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1fc80a1b40 0x7f1fc80a4000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fdc103840 0x7f1fdc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fdc101ca0 0x7f1fdc197d30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 --2- 172.21.15.106:0/1013745612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fdc102ea0 0x7f1fdc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 >> 172.21.15.106:0/1013745612 conn(0x7f1fdc0fd280 msgr2=0x7f1fdc1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 shutdown_connections 2024-03-11T19:18:14.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-11T19:18:14.280+0000 7f1fe42fd700 1 -- 172.21.15.106:0/1013745612 wait complete. 2024-03-11T19:18:14.959 DEBUG:teuthology.orchestra.run.smithi106:osd.10> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.10.service 2024-03-11T19:18:14.962 INFO:tasks.cephadm:Deploying osd.11 on smithi143 with /dev/vg_nvme/lv_1... 2024-03-11T19:18:14.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:18:15.211 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:18:15.332 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.333 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:15 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:15 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:15 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-mon[35889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:18:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-mon[35889]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-mon[35889]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:18:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:16.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-mon[35889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:16.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:18:16.265+0000 7f820ebf1700 -1 osd.10 0 waiting for initial osdmap 2024-03-11T19:18:16.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:18:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:18:16.276+0000 7f820a9e4700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:18:16.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:16 smithi143 ceph-mon[36013]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:18:16.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:16 smithi143 ceph-mon[36013]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:16.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:16 smithi143 ceph-mon[36013]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:18:16.669 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:16 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:16.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:16 smithi143 ceph-mon[36013]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:16 smithi072 ceph-mon[31695]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:18:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:16 smithi072 ceph-mon[31695]: from='osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:16 smithi072 ceph-mon[31695]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:18:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:16 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:16 smithi072 ceph-mon[31695]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-11T19:18:17.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:17 smithi106 ceph-mon[35889]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:17.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:17 smithi106 ceph-mon[35889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:18:17.273 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:17 smithi106 ceph-mon[35889]: osdmap e72: 11 total, 10 up, 11 in 2024-03-11T19:18:17.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:18:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:18:17.390 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-03-11T19:18:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-03-11T19:18:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121302 s, 864 MB/s 2024-03-11T19:18:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-03-11T19:18:17.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:17 smithi072 ceph-mon[31695]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:17.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:17 smithi072 ceph-mon[31695]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:18:17.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:17 smithi072 ceph-mon[31695]: osdmap e72: 11 total, 10 up, 11 in 2024-03-11T19:18:17.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:17.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:17 smithi143 ceph-mon[36013]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:18:17.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:17 smithi143 ceph-mon[36013]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-11T19:18:17.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:17 smithi143 ceph-mon[36013]: osdmap e72: 11 total, 10 up, 11 in 2024-03-11T19:18:17.688 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:18.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-03-11T19:18:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:18:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:18 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:18:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:18 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:18:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:18 smithi143 ceph-mon[36013]: osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] boot 2024-03-11T19:18:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:18 smithi143 ceph-mon[36013]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:18:18.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:18 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:18.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:18 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:18:18.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:18 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:18:18.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:18 smithi106 ceph-mon[35889]: osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] boot 2024-03-11T19:18:18.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:18 smithi106 ceph-mon[35889]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:18:18.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:18 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:18 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:18:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:18 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:18:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:18 smithi072 ceph-mon[31695]: osd.10 [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] boot 2024-03-11T19:18:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:18 smithi072 ceph-mon[31695]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:18:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:18 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:18:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi106 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:19 smithi143 ceph-mon[36013]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:18:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi106 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:19.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:19 smithi106 ceph-mon[35889]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:18:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi106 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:19.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:19.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:19.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:19.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:19 smithi072 ceph-mon[31695]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:18:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 -- 172.21.15.143:0/4042581821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e8103040 msgr2=0x7f42e81034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 --2- 172.21.15.143:0/4042581821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e8103040 0x7f42e81034a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009990 tx=0x7f42dc009ca0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 -- 172.21.15.143:0/4042581821 shutdown_connections 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 --2- 172.21.15.143:0/4042581821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e81039e0 0x7f42e810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 --2- 172.21.15.143:0/4042581821 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42e80fffc0 0x7f42e81003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 --2- 172.21.15.143:0/4042581821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e8103040 0x7f42e81034a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 -- 172.21.15.143:0/4042581821 >> 172.21.15.143:0/4042581821 conn(0x7f42e80fb900 msgr2=0x7f42e80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 -- 172.21.15.143:0/4042581821 shutdown_connections 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.077+0000 7f42ee755700 1 -- 172.21.15.143:0/4042581821 wait complete. 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 Processor -- start 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 -- start start 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42e80fffc0 0x7f42e8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 0x7f42e81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819f830 con 0x7f42e80fffc0 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819f970 con 0x7f42e8103040 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42ee755700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819fb30 con 0x7f42e81039e0 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42eccf2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42e77fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 0x7f42e81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42e77fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 0x7f42e81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.143:52820/0 (socket says 172.21.15.143:52820) 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42eccf2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52558/0 (socket says 172.21.15.143:52558) 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.078+0000 7f42e77fe700 1 -- 172.21.15.143:0/48123623 learned_addr learned my addr 172.21.15.143:0/48123623 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 -- 172.21.15.143:0/48123623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 msgr2=0x7f42e81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 0x7f42e81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 -- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42e80fffc0 msgr2=0x7f42e8197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42e80fffc0 0x7f42e8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 -- 172.21.15.143:0/48123623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0095f0 con 0x7f42e81039e0 2024-03-11T19:18:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42eccf2700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007f10 tx=0x7f42e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e000ddd0 con 0x7f42e81039e0 2024-03-11T19:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e819fe10 con 0x7f42e81039e0 2024-03-11T19:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e000f040 con 0x7f42e81039e0 2024-03-11T19:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0015400 con 0x7f42e81039e0 2024-03-11T19:18:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81a0280 con 0x7f42e81039e0 2024-03-11T19:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.080+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8109200 con 0x7f42e81039e0 2024-03-11T19:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.082+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f42e0004ad0 con 0x7f42e81039e0 2024-03-11T19:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.082+0000 7f42e57fa700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 0x7f42d00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.083+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f42e00ccea0 con 0x7f42e81039e0 2024-03-11T19:18:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.083+0000 7f42e7fff700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 0x7f42d00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.084+0000 7f42e7fff700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 0x7f42d00a3f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42e8198b90 tx=0x7f42d8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.086+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42e00966e0 con 0x7f42e81039e0 2024-03-11T19:18:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:20.290+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e802d7d0 con 0x7f42d00a1ac0 2024-03-11T19:18:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:20 smithi106 ceph-mon[35889]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:20 smithi106 ceph-mon[35889]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:18:20.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:20.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:20.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:20.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:20 smithi072 ceph-mon[31695]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:20.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:20 smithi072 ceph-mon[31695]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:18:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:20 smithi143 ceph-mon[36013]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:20 smithi143 ceph-mon[36013]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:18:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:21 smithi106 ceph-mon[35889]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:18:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:18:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:21 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:21 smithi106 ceph-mon[35889]: osdmap e77: 11 total, 11 up, 11 in 2024-03-11T19:18:21.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:21 smithi143 ceph-mon[36013]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:21.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:18:21.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:18:21.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:21 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:21.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:21 smithi143 ceph-mon[36013]: osdmap e77: 11 total, 11 up, 11 in 2024-03-11T19:18:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:21 smithi072 ceph-mon[31695]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:18:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:18:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:21 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:21 smithi072 ceph-mon[31695]: osdmap e77: 11 total, 11 up, 11 in 2024-03-11T19:18:22.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:22 smithi106 ceph-mon[35889]: pgmap v273: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:22.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:22 smithi072 ceph-mon[31695]: pgmap v273: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:22.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:22 smithi143 ceph-mon[36013]: pgmap v273: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:23 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/114754794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:23 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:23 smithi106 ceph-mon[35889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]': finished 2024-03-11T19:18:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:23 smithi106 ceph-mon[35889]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:18:23.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:23 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/114754794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:23 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:23 smithi072 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]': finished 2024-03-11T19:18:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:23 smithi072 ceph-mon[31695]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:18:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:23 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/114754794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:23 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]: dispatch 2024-03-11T19:18:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:23 smithi143 ceph-mon[36013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbf1e6e6-c003-4c61-a3d0-a1760ed65098"}]': finished 2024-03-11T19:18:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:23 smithi143 ceph-mon[36013]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:18:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:24.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:24 smithi106 ceph-mon[35889]: from='client.? 172.21.15.143:0/1479198063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:18:24.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:24 smithi106 ceph-mon[35889]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-11T19:18:24.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:24 smithi072 ceph-mon[31695]: from='client.? 172.21.15.143:0/1479198063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:18:24.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:24 smithi072 ceph-mon[31695]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-11T19:18:24.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:24 smithi143 ceph-mon[36013]: from='client.? 172.21.15.143:0/1479198063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:18:24.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:24 smithi143 ceph-mon[36013]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-11T19:18:27.106 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:26 smithi143 ceph-mon[36013]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:18:27.383 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:26 smithi072 ceph-mon[31695]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:18:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:26 smithi106 ceph-mon[35889]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:18:29.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:29 smithi106 ceph-mon[35889]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-11T19:18:29.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:29 smithi072 ceph-mon[31695]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-11T19:18:29.433 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:29 smithi143 ceph-mon[36013]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-11T19:18:31.351 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:31 smithi143 ceph-mon[36013]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:18:31.351 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:18:31.351 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:31 smithi106 ceph-mon[35889]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:18:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:18:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:31 smithi072 ceph-mon[31695]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:18:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:18:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:32.354 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:32 smithi143 ceph-mon[36013]: Deploying daemon osd.11 on smithi143 2024-03-11T19:18:32.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:32 smithi106 ceph-mon[35889]: Deploying daemon osd.11 on smithi143 2024-03-11T19:18:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:32 smithi072 ceph-mon[31695]: Deploying daemon osd.11 on smithi143 2024-03-11T19:18:33.354 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:33 smithi143 ceph-mon[36013]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:18:33.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:33 smithi106 ceph-mon[35889]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:18:33.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:33 smithi072 ceph-mon[31695]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:18:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:35 smithi106 ceph-mon[35889]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:35.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:35 smithi072 ceph-mon[31695]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:35 smithi143 ceph-mon[36013]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:37.380 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:37 smithi143 ceph-mon[36013]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:37.381 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.381 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.381 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:37 smithi106 ceph-mon[35889]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:37.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:37 smithi072 ceph-mon[31695]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:37.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:37.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:39.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:39 smithi106 ceph-mon[35889]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:39 smithi072 ceph-mon[31695]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:39 smithi143 ceph-mon[36013]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:40.080 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 11 on host 'smithi143' 2024-03-11T19:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.076+0000 7f42e57fa700 1 -- 172.21.15.143:0/48123623 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f42e802d7d0 con 0x7f42d00a1ac0 2024-03-11T19:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 msgr2=0x7f42d00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 0x7f42d00a3f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42e8198b90 tx=0x7f42d8006d20 comp rx=0 tx=0).stop 2024-03-11T19:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 msgr2=0x7f42e819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007f10 tx=0x7f42e000d3b0 comp rx=0 tx=0).stop 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 shutdown_connections 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f42d00a1ac0 0x7f42d00a3f80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42e81039e0 0x7f42e819f260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42e80fffc0 0x7f42e8197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 --2- 172.21.15.143:0/48123623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42e8103040 0x7f42e81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 >> 172.21.15.143:0/48123623 conn(0x7f42e80fb900 msgr2=0x7f42e8106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 shutdown_connections 2024-03-11T19:18:40.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:40.079+0000 7f42ee755700 1 -- 172.21.15.143:0/48123623 wait complete. 2024-03-11T19:18:40.742 DEBUG:teuthology.orchestra.run.smithi143:osd.11> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.11.service 2024-03-11T19:18:40.744 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-11T19:18:40.744 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.193 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:18:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:18:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:41.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:42.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:42 smithi106 ceph-mon[35889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:18:42.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:42 smithi106 ceph-mon[35889]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:42 smithi106 ceph-mon[35889]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:18:42.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:42.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:42 smithi106 ceph-mon[35889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:42 smithi072 ceph-mon[31695]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:18:42.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:42 smithi072 ceph-mon[31695]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:42 smithi072 ceph-mon[31695]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:18:42.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:42.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:42 smithi072 ceph-mon[31695]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-mon[36013]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:18:42.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-mon[36013]: from='osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-mon[36013]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:18:42.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:42.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-mon[36013]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-03-11T19:18:42.804 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:18:42.450+0000 7fd3247b7700 -1 osd.11 0 waiting for initial osdmap 2024-03-11T19:18:42.804 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:18:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:18:42.464+0000 7fd31c5a2700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:18:43.311 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:43 smithi072 ceph-mon[31695]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:43.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:43 smithi072 ceph-mon[31695]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:18:43.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:43 smithi072 ceph-mon[31695]: osdmap e80: 12 total, 11 up, 12 in 2024-03-11T19:18:43.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:43.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:43 smithi072 ceph-mon[31695]: from='osd.11 ' entity='osd.11' 2024-03-11T19:18:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:43 smithi106 ceph-mon[35889]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:43 smithi106 ceph-mon[35889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:18:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:43 smithi106 ceph-mon[35889]: osdmap e80: 12 total, 11 up, 12 in 2024-03-11T19:18:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:43.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:43 smithi106 ceph-mon[35889]: from='osd.11 ' entity='osd.11' 2024-03-11T19:18:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:43 smithi143 ceph-mon[36013]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:43 smithi143 ceph-mon[36013]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-03-11T19:18:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:43 smithi143 ceph-mon[36013]: osdmap e80: 12 total, 11 up, 12 in 2024-03-11T19:18:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:43 smithi143 ceph-mon[36013]: from='osd.11 ' entity='osd.11' 2024-03-11T19:18:43.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.575+0000 7f720d200700 1 -- 172.21.15.72:0/1667112692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f72080746d0 msgr2=0x7f7208074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.575+0000 7f720d200700 1 --2- 172.21.15.72:0/1667112692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f72080746d0 0x7f7208074b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009990 tx=0x7f71f4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 -- 172.21.15.72:0/1667112692 shutdown_connections 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 --2- 172.21.15.72:0/1667112692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7208109b80 0x7f720810c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 --2- 172.21.15.72:0/1667112692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f7208074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 --2- 172.21.15.72:0/1667112692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f72080746d0 0x7f7208074b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 -- 172.21.15.72:0/1667112692 >> 172.21.15.72:0/1667112692 conn(0x7f72080fd210 msgr2=0x7f72080ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 -- 172.21.15.72:0/1667112692 shutdown_connections 2024-03-11T19:18:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.576+0000 7f720d200700 1 -- 172.21.15.72:0/1667112692 wait complete. 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.577+0000 7f720d200700 1 Processor -- start 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.577+0000 7f720d200700 1 -- start start 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 0x7f7208199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 0x7f72081a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72081a19f0 con 0x7f7208109b80 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72081a1b30 con 0x7f72080746d0 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f720d200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72081a1cc0 con 0x7f7208075ca0 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f7205400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f7205400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:52180/0 (socket says 172.21.15.72:52180) 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f7205400700 1 -- 172.21.15.72:0/468228686 learned_addr learned my addr 172.21.15.72:0/468228686 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f7205e00700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 0x7f7208199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.578+0000 7f7206800700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 0x7f72081a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 -- 172.21.15.72:0/468228686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 msgr2=0x7f7208199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 0x7f7208199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 -- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 msgr2=0x7f72081a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 0x7f72081a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 -- 172.21.15.72:0/468228686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40095f0 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7206800700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 0x7f72081a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205e00700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 0x7f7208199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f7205400700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71f400b650 tx=0x7f71f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f401d070 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f400be80 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72081a1f40 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.579+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4017650 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.580+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72081a22a0 con 0x7f7208075ca0 2024-03-11T19:18:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.581+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208108940 con 0x7f7208075ca0 2024-03-11T19:18:43.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.584+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f71f4021410 con 0x7f7208075ca0 2024-03-11T19:18:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.585+0000 7f71fea00700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 0x7f71ec0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.585+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71f40ce4e0 con 0x7f7208075ca0 2024-03-11T19:18:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.585+0000 7f7205e00700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 0x7f71ec0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:43.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.586+0000 7f7205e00700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 0x7f71ec0a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7208076080 tx=0x7f71f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:43.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.589+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71f4097b80 con 0x7f7208075ca0 2024-03-11T19:18:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.811+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f720810f810 con 0x7f7208075ca0 2024-03-11T19:18:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.812+0000 7f71fea00700 1 -- 172.21.15.72:0/468228686 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f71f409b240 con 0x7f7208075ca0 2024-03-11T19:18:43.815 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:18:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 msgr2=0x7f71ec0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 0x7f71ec0a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7208076080 tx=0x7f71f8009260 comp rx=0 tx=0).stop 2024-03-11T19:18:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 msgr2=0x7f720819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71f400b650 tx=0x7f71f400f690 comp rx=0 tx=0).stop 2024-03-11T19:18:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 shutdown_connections 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f71ec0a1ac0 0x7f71ec0a3f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7208109b80 0x7f72081a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72080746d0 0x7f7208199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 --2- 172.21.15.72:0/468228686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7208075ca0 0x7f720819a3a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 >> 172.21.15.72:0/468228686 conn(0x7f72080fd210 msgr2=0x7f7208105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 shutdown_connections 2024-03-11T19:18:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:43.815+0000 7f720d200700 1 -- 172.21.15.72:0/468228686 wait complete. 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: purged_snaps scrub starts 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: purged_snaps scrub ok 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] boot 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:44 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/468228686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: purged_snaps scrub starts 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: purged_snaps scrub ok 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] boot 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:44 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/468228686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: purged_snaps scrub starts 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: purged_snaps scrub ok 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: osd.11 [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] boot 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:18:44.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:44 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/468228686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:18:44.872 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1710184723,"num_in_osds":12,"osd_in_since":1710184703,"num_remapped_pgs":0} 2024-03-11T19:18:44.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:18:45.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:45.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:45.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:45.312 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:45.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:45.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:45.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: Detected new or changed devices on smithi143 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 4270M 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:46.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:46 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:46.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: Detected new or changed devices on smithi143 2024-03-11T19:18:46.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 4270M 2024-03-11T19:18:46.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:18:46.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:46.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:46.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:46 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: Detected new or changed devices on smithi143 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 4270M 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:46.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:46 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:47.154 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:47 smithi072 ceph-mon[31695]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:47.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:47 smithi106 ceph-mon[35889]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:47 smithi143 ceph-mon[36013]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 -- 172.21.15.72:0/1049677928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec106ff0 msgr2=0x7f2dec10eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 --2- 172.21.15.72:0/1049677928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec106ff0 0x7f2dec10eb70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b0a0 tx=0x7f2de400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 -- 172.21.15.72:0/1049677928 shutdown_connections 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 --2- 172.21.15.72:0/1049677928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec106ff0 0x7f2dec10eb70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 --2- 172.21.15.72:0/1049677928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec06a410 0x7f2dec06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 --2- 172.21.15.72:0/1049677928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06adc0 0x7f2dec1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 -- 172.21.15.72:0/1049677928 >> 172.21.15.72:0/1049677928 conn(0x7f2dec076bc0 msgr2=0x7f2dec076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:47.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.678+0000 7f2df4000700 1 -- 172.21.15.72:0/1049677928 shutdown_connections 2024-03-11T19:18:47.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.679+0000 7f2df4000700 1 -- 172.21.15.72:0/1049677928 wait complete. 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 Processor -- start 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 -- start start 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 0x7f2dec19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec106ff0 0x7f2dec19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19f440 con 0x7f2dec06adc0 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19f5b0 con 0x7f2dec06a410 2024-03-11T19:18:47.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.680+0000 7f2df4000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec19f770 con 0x7f2dec106ff0 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df1600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 0x7f2dec19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42702/0 (socket says 172.21.15.72:42702) 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 -- 172.21.15.72:0/974577387 learned_addr learned my addr 172.21.15.72:0/974577387 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df2000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec106ff0 0x7f2dec19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 -- 172.21.15.72:0/974577387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec106ff0 msgr2=0x7f2dec19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec106ff0 0x7f2dec19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 -- 172.21.15.72:0/974577387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 msgr2=0x7f2dec19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 0x7f2dec19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 -- 172.21.15.72:0/974577387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4009d00 con 0x7f2dec06adc0 2024-03-11T19:18:47.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df1600700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 0x7f2dec19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:18:47.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.681+0000 7f2df0c00700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e940 tx=0x7f2dd800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:47.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.682+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8009730 con 0x7f2dec06adc0 2024-03-11T19:18:47.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.682+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec19fa50 con 0x7f2dec06adc0 2024-03-11T19:18:47.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.682+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dd8010430 con 0x7f2dec06adc0 2024-03-11T19:18:47.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.682+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8018660 con 0x7f2dec06adc0 2024-03-11T19:18:47.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.682+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec1a72b0 con 0x7f2dec06adc0 2024-03-11T19:18:47.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.683+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec1001b0 con 0x7f2dec06adc0 2024-03-11T19:18:47.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.685+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2dd80108a0 con 0x7f2dec06adc0 2024-03-11T19:18:47.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.687+0000 7f2dea000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 0x7f2dd40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:47.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.687+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2dd8014070 con 0x7f2dec06adc0 2024-03-11T19:18:47.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.687+0000 7f2df1600700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 0x7f2dd40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:47.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.688+0000 7f2df1600700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 0x7f2dd40a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011490 tx=0x7f2de0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:47.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.690+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2dd8096b70 con 0x7f2dec06adc0 2024-03-11T19:18:47.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.908+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dec19b8c0 con 0x7f2dec06adc0 2024-03-11T19:18:47.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.910+0000 7f2dea000700 1 -- 172.21.15.72:0/974577387 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18469 (secure 0 0 0) 0x7f2dd809a230 con 0x7f2dec06adc0 2024-03-11T19:18:47.914 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:18:47.914 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":82,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:18:45.053935+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:18:47.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.913+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 msgr2=0x7f2dd40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:47.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.913+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 0x7f2dd40a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de0011490 tx=0x7f2de0009260 comp rx=0 tx=0).stop 2024-03-11T19:18:47.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.913+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 msgr2=0x7f2dec19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:47.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.913+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e940 tx=0x7f2dd800ec50 comp rx=0 tx=0).stop 2024-03-11T19:18:47.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 shutdown_connections 2024-03-11T19:18:47.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2dd40a1b40 0x7f2dd40a4000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dec106ff0 0x7f2dec19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dec06a410 0x7f2dec19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 --2- 172.21.15.72:0/974577387 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dec06adc0 0x7f2dec19aaa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:47.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 >> 172.21.15.72:0/974577387 conn(0x7f2dec076bc0 msgr2=0x7f2dec10d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:47.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 shutdown_connections 2024-03-11T19:18:47.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:47.914+0000 7f2df4000700 1 -- 172.21.15.72:0/974577387 wait complete. 2024-03-11T19:18:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/974577387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:48.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/974577387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:48.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/974577387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:48.926 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:18:48.926 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-11T19:18:49.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:49 smithi072 ceph-mon[31695]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:18:49.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:49 smithi106 ceph-mon[35889]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:49 smithi143 ceph-mon[36013]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:51.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:51 smithi106 ceph-mon[35889]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:51.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:51 smithi072 ceph-mon[31695]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:51.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:51 smithi143 ceph-mon[36013]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 -- 172.21.15.72:0/921020805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c10d620 msgr2=0x7f702c1151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 --2- 172.21.15.72:0/921020805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c10d620 0x7f702c1151f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7024009f90 tx=0x7f702400b330 comp rx=0 tx=0).stop 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 -- 172.21.15.72:0/921020805 shutdown_connections 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 --2- 172.21.15.72:0/921020805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c10d620 0x7f702c1151f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 --2- 172.21.15.72:0/921020805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 --2- 172.21.15.72:0/921020805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c108ef0 0x7f702c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.603+0000 7f7032400700 1 -- 172.21.15.72:0/921020805 >> 172.21.15.72:0/921020805 conn(0x7f702c0762c0 msgr2=0x7f702c0786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:51.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.604+0000 7f7032400700 1 -- 172.21.15.72:0/921020805 shutdown_connections 2024-03-11T19:18:51.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.604+0000 7f7032400700 1 -- 172.21.15.72:0/921020805 wait complete. 2024-03-11T19:18:51.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.605+0000 7f7032400700 1 Processor -- start 2024-03-11T19:18:51.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.605+0000 7f7032400700 1 -- start start 2024-03-11T19:18:51.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c108ef0 0x7f702c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c19eeb0 con 0x7f702c108ef0 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c19f020 con 0x7f702c10d620 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f7032400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c19f1e0 con 0x7f702c108540 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c108ef0 0x7f702c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702b400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40832/0 (socket says 172.21.15.72:40832) 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 -- 172.21.15.72:0/1238500941 learned_addr learned my addr 172.21.15.72:0/1238500941 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 -- 172.21.15.72:0/1238500941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 msgr2=0x7f702c1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 -- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c108ef0 msgr2=0x7f702c1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:51.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c108ef0 0x7f702c1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702be00700 1 -- 172.21.15.72:0/1238500941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009c40 con 0x7f702c10d620 2024-03-11T19:18:51.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.606+0000 7f702b400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:18:51.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f702be00700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f70240094a0 tx=0x7f70240094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400e040 con 0x7f702c10d620 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f702400be60 con 0x7f702c10d620 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240052a0 con 0x7f702c10d620 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c19f460 con 0x7f702c10d620 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.607+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c1ab6b0 con 0x7f702c10d620 2024-03-11T19:18:51.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.608+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c110940 con 0x7f702c10d620 2024-03-11T19:18:51.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.610+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f702401d470 con 0x7f702c10d620 2024-03-11T19:18:51.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.611+0000 7f7023e00700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 0x7f70100a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:51.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.611+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70240ce520 con 0x7f702c10d620 2024-03-11T19:18:51.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.612+0000 7f702b400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 0x7f70100a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:51.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.612+0000 7f702b400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 0x7f70100a4000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f701c011490 tx=0x7f701c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:51.618 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.614+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7024097bc0 con 0x7f702c10d620 2024-03-11T19:18:51.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.835+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f702c19fd30 con 0x7f702c10d620 2024-03-11T19:18:51.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.836+0000 7f7023e00700 1 -- 172.21.15.72:0/1238500941 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f702409b280 con 0x7f702c10d620 2024-03-11T19:18:51.840 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:18:51.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 msgr2=0x7f70100a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:51.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 0x7f70100a4000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f701c011490 tx=0x7f701c009260 comp rx=0 tx=0).stop 2024-03-11T19:18:51.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 msgr2=0x7f702c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:51.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f70240094a0 tx=0x7f70240094d0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 shutdown_connections 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f70100a1b40 0x7f70100a4000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702c10d620 0x7f702c19e800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f702c108540 0x7f702c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 --2- 172.21.15.72:0/1238500941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f702c108ef0 0x7f702c1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.840+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 >> 172.21.15.72:0/1238500941 conn(0x7f702c0762c0 msgr2=0x7f702c076e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.841+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 shutdown_connections 2024-03-11T19:18:51.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:51.841+0000 7f7032400700 1 -- 172.21.15.72:0/1238500941 wait complete. 2024-03-11T19:18:52.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:52 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1238500941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:52.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:52 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1238500941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:52.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:52 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1238500941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.a on smithi072 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.d on smithi072 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.g on smithi072 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.j on smithi072 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.b on smithi106 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.e on smithi106 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.h on smithi106 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.k on smithi106 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.c on smithi143 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.f on smithi143 2024-03-11T19:18:52.690 INFO:tasks.cephadm:Adding mds.i on smithi143 2024-03-11T19:18:52.691 INFO:tasks.cephadm:Adding mds.l on smithi143 2024-03-11T19:18:52.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph orch apply mds all '12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l' 2024-03-11T19:18:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:18:53.354 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:53 smithi143 ceph-mon[36013]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:53.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:53 smithi106 ceph-mon[35889]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:53 smithi072 ceph-mon[31695]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:54.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 -- 172.21.15.143:0/1212504705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 msgr2=0x7fe51c104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:54.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 --2- 172.21.15.143:0/1212504705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c104bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe510009990 tx=0x7fe510009ca0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 -- 172.21.15.143:0/1212504705 shutdown_connections 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 --2- 172.21.15.143:0/1212504705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe51c107a30 0x7fe51c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 --2- 172.21.15.143:0/1212504705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c104bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 --2- 172.21.15.143:0/1212504705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 -- 172.21.15.143:0/1212504705 >> 172.21.15.143:0/1212504705 conn(0x7fe51c0fc160 msgr2=0x7fe51c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 -- 172.21.15.143:0/1212504705 shutdown_connections 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.739+0000 7fe5236a7700 1 -- 172.21.15.143:0/1212504705 wait complete. 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 Processor -- start 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 -- start start 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe51c107a30 0x7fe51c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19f7b0 con 0x7fe51c107a30 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19f8f0 con 0x7fe51c105100 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe5236a7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19fab0 con 0x7fe51c1027d0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe521443700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe521443700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58878/0 (socket says 172.21.15.143:58878) 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.740+0000 7fe521443700 1 -- 172.21.15.143:0/4284834318 learned_addr learned my addr 172.21.15.143:0/4284834318 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe520c42700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 -- 172.21.15.143:0/4284834318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 msgr2=0x7fe51c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 -- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe51c107a30 msgr2=0x7fe51c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe51c107a30 0x7fe51c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 -- 172.21.15.143:0/4284834318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100095f0 con 0x7fe51c1027d0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe520c42700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe521443700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b5c0 tx=0x7fe51000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51001d070 con 0x7fe51c1027d0 2024-03-11T19:18:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c19fd30 con 0x7fe51c1027d0 2024-03-11T19:18:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe51000fc30 con 0x7fe51c1027d0 2024-03-11T19:18:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510017650 con 0x7fe51c1027d0 2024-03-11T19:18:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.741+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c1a0140 con 0x7fe51c1027d0 2024-03-11T19:18:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.742+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c068970 con 0x7fe51c1027d0 2024-03-11T19:18:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.743+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe5100177b0 con 0x7fe51c1027d0 2024-03-11T19:18:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.744+0000 7fe50e7fc700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 0x7fe5080a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.744+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5100ceb60 con 0x7fe51c1027d0 2024-03-11T19:18:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.744+0000 7fe520c42700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 0x7fe5080a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.745+0000 7fe520c42700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 0x7fe5080a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe518005950 tx=0x7fe5180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:54.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.747+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5100982a0 con 0x7fe51c1027d0 2024-03-11T19:18:54.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.957+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe51c02dad0 con 0x7fe5080a1ac0 2024-03-11T19:18:54.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.974+0000 7fe50e7fc700 1 -- 172.21.15.143:0/4284834318 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe51c02dad0 con 0x7fe5080a1ac0 2024-03-11T19:18:54.979 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mds.all update... 2024-03-11T19:18:54.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 msgr2=0x7fe5080a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 0x7fe5080a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe518005950 tx=0x7fe5180058e0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 msgr2=0x7fe51c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b5c0 tx=0x7fe51000f740 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 shutdown_connections 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5080a1ac0 0x7fe5080a3f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe51c107a30 0x7fe51c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe51c1027d0 0x7fe51c197c60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 --2- 172.21.15.143:0/4284834318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c105100 0x7fe51c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 >> 172.21.15.143:0/4284834318 conn(0x7fe51c0fc160 msgr2=0x7fe51c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 shutdown_connections 2024-03-11T19:18:54.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-03-11T19:18:54.976+0000 7fe5236a7700 1 -- 172.21.15.143:0/4284834318 wait complete. 2024-03-11T19:18:55.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:55 smithi143 ceph-mon[36013]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:55.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:55.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:55 smithi106 ceph-mon[35889]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:55 smithi072 ceph-mon[31695]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:55.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:18:55.765 DEBUG:teuthology.orchestra.run.smithi072:mds.a> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.a.service 2024-03-11T19:18:55.772 DEBUG:teuthology.orchestra.run.smithi072:mds.d> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.d.service 2024-03-11T19:18:55.817 DEBUG:teuthology.orchestra.run.smithi072:mds.g> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.g.service 2024-03-11T19:18:55.865 DEBUG:teuthology.orchestra.run.smithi072:mds.j> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.j.service 2024-03-11T19:18:55.911 DEBUG:teuthology.orchestra.run.smithi106:mds.b> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.b.service 2024-03-11T19:18:55.913 DEBUG:teuthology.orchestra.run.smithi106:mds.e> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.e.service 2024-03-11T19:18:55.956 DEBUG:teuthology.orchestra.run.smithi106:mds.h> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.h.service 2024-03-11T19:18:55.999 DEBUG:teuthology.orchestra.run.smithi106:mds.k> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.k.service 2024-03-11T19:18:56.042 DEBUG:teuthology.orchestra.run.smithi143:mds.c> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.c.service 2024-03-11T19:18:56.045 DEBUG:teuthology.orchestra.run.smithi143:mds.f> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.f.service 2024-03-11T19:18:56.088 DEBUG:teuthology.orchestra.run.smithi143:mds.i> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.i.service 2024-03-11T19:18:56.131 DEBUG:teuthology.orchestra.run.smithi143:mds.l> sudo journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.l.service 2024-03-11T19:18:56.174 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-11T19:18:56.174 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 -- 172.21.15.72:0/802310229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 msgr2=0x7ff60c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 --2- 172.21.15.72:0/802310229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c108e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f802bc40 comp rx=0 tx=0).stop 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 -- 172.21.15.72:0/802310229 shutdown_connections 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 --2- 172.21.15.72:0/802310229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c109370 0x7ff60c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 --2- 172.21.15.72:0/802310229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c108e30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 --2- 172.21.15.72:0/802310229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c1077c0 0x7ff60c107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 -- 172.21.15.72:0/802310229 >> 172.21.15.72:0/802310229 conn(0x7ff60c076790 msgr2=0x7ff60c078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:56.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 -- 172.21.15.72:0/802310229 shutdown_connections 2024-03-11T19:18:56.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.387+0000 7ff613c00640 1 -- 172.21.15.72:0/802310229 wait complete. 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.388+0000 7ff613c00640 1 Processor -- start 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.388+0000 7ff613c00640 1 -- start start 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.388+0000 7ff613c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c1077c0 0x7ff60c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.388+0000 7ff613c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff613c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c109370 0x7ff60c1a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff613c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a5340 con 0x7ff60c109370 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff613c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a54b0 con 0x7ff60c1077c0 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff613c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a5670 con 0x7ff60c1089b0 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:52714/0 (socket says 172.21.15.72:52714) 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 -- 172.21.15.72:0/2708077175 learned_addr learned my addr 172.21.15.72:0/2708077175 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:56.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff611200640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c1077c0 0x7ff60c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff611c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c109370 0x7ff60c1a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 -- 172.21.15.72:0/2708077175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c1077c0 msgr2=0x7ff60c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c1077c0 0x7ff60c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 -- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c109370 msgr2=0x7ff60c1a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c109370 0x7ff60c1a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 -- 172.21.15.72:0/2708077175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604009840 con 0x7ff60c1089b0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.389+0000 7ff60be00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8006010 tx=0x7ff5f8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.390+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80043d0 con 0x7ff60c1089b0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.390+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5f80052d0 con 0x7ff60c1089b0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.390+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8003bc0 con 0x7ff60c1089b0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.390+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f8009560 con 0x7ff60c1089b0 2024-03-11T19:18:56.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.390+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c1a5cc0 con 0x7ff60c1089b0 2024-03-11T19:18:56.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:56 smithi106 ceph-mon[35889]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:56.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:56 smithi106 ceph-mon[35889]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-03-11T19:18:56.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.391+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c10ebb0 con 0x7ff60c1089b0 2024-03-11T19:18:56.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.393+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff5f8003d40 con 0x7ff60c1089b0 2024-03-11T19:18:56.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.394+0000 7ff609600640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 0x7ff5f40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.394+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5f80ef1b0 con 0x7ff60c1089b0 2024-03-11T19:18:56.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.394+0000 7ff611200640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 0x7ff5f40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.395+0000 7ff611200640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 0x7ff5f40a3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff60c108810 tx=0x7ff600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:56.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.398+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5f80b87c0 con 0x7ff60c1089b0 2024-03-11T19:18:56.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:56 smithi072 ceph-mon[31695]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:56.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:56 smithi072 ceph-mon[31695]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-03-11T19:18:56.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:56 smithi143 ceph-mon[36013]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:18:56.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:56 smithi143 ceph-mon[36013]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-03-11T19:18:56.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.596+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff60c10e620 con 0x7ff60c1089b0 2024-03-11T19:18:56.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.598+0000 7ff609600640 1 -- 172.21.15.72:0/2708077175 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18469 (secure 0 0 0) 0x7ff5f80bbe80 con 0x7ff60c1089b0 2024-03-11T19:18:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:18:56.602 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":82,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:18:45.053935+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 msgr2=0x7ff5f40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 0x7ff5f40a3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff60c108810 tx=0x7ff600005f90 comp rx=0 tx=0).stop 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 msgr2=0x7ff60c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8006010 tx=0x7ff5f8005e80 comp rx=0 tx=0).stop 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 shutdown_connections 2024-03-11T19:18:56.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff5f40a1ab0 0x7ff5f40a3f70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff60c109370 0x7ff60c1a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff60c1089b0 0x7ff60c19dde0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 --2- 172.21.15.72:0/2708077175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60c1077c0 0x7ff60c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 >> 172.21.15.72:0/2708077175 conn(0x7ff60c076790 msgr2=0x7ff60c10bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 shutdown_connections 2024-03-11T19:18:56.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.601+0000 7ff613c00640 1 -- 172.21.15.72:0/2708077175 wait complete. 2024-03-11T19:18:56.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:18:56.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 -- 172.21.15.72:0/1386948605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 msgr2=0x7fd03010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 --2- 172.21.15.72:0/1386948605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd03010fbe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd028009ca0 tx=0x7fd02802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 -- 172.21.15.72:0/1386948605 shutdown_connections 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 --2- 172.21.15.72:0/1386948605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd03010fbe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 --2- 172.21.15.72:0/1386948605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd030108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 --2- 172.21.15.72:0/1386948605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 0x7fd030107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.836+0000 7fd034c00640 1 -- 172.21.15.72:0/1386948605 >> 172.21.15.72:0/1386948605 conn(0x7fd030076740 msgr2=0x7fd030078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.837+0000 7fd034c00640 1 -- 172.21.15.72:0/1386948605 shutdown_connections 2024-03-11T19:18:56.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.837+0000 7fd034c00640 1 -- 172.21.15.72:0/1386948605 wait complete. 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.837+0000 7fd034c00640 1 Processor -- start 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 -- start start 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 0x7fd03019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd03019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0301a52d0 con 0x7fd030109350 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0301a5440 con 0x7fd0301077a0 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd034c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0301a5600 con 0x7fd030108990 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd02e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd02e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52194/0 (socket says 172.21.15.72:52194) 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd02e800640 1 -- 172.21.15.72:0/2180436655 learned_addr learned my addr 172.21.15.72:0/2180436655 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd025400640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd03019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.838+0000 7fd02de00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 0x7fd03019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 -- 172.21.15.72:0/2180436655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 msgr2=0x7fd03019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 0x7fd03019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 -- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 msgr2=0x7fd0301a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 -- 172.21.15.72:0/2180436655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028009900 con 0x7fd030108990 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd02e800640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd025400640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd03019dd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd01800e9f0 tx=0x7fd01800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800ccc0 con 0x7fd030108990 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd01800ce20 con 0x7fd030108990 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0301a58e0 con 0x7fd030108990 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018004a00 con 0x7fd030108990 2024-03-11T19:18:56.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.839+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0301a5da0 con 0x7fd030108990 2024-03-11T19:18:56.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.840+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03010ea60 con 0x7fd030108990 2024-03-11T19:18:56.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.842+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd0180050b0 con 0x7fd030108990 2024-03-11T19:18:56.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.843+0000 7fd027400640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 0x7fd0140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:56.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.844+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd018014070 con 0x7fd030108990 2024-03-11T19:18:56.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.844+0000 7fd02de00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 0x7fd0140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:56.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.844+0000 7fd02de00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 0x7fd0140a3ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0301087f0 tx=0x7fd020005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:56.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:56.846+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd018096540 con 0x7fd030108990 2024-03-11T19:18:57.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.044+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd030113330 con 0x7fd030108990 2024-03-11T19:18:57.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.044+0000 7fd027400640 1 -- 172.21.15.72:0/2180436655 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd030113330 con 0x7fd030108990 2024-03-11T19:18:57.048 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:18:57.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 msgr2=0x7fd0140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 0x7fd0140a3ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0301087f0 tx=0x7fd020005f90 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 msgr2=0x7fd03019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd03019dd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd01800e9f0 tx=0x7fd01800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 shutdown_connections 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0140a1a30 0x7fd0140a3ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030109350 0x7fd0301a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd030108990 0x7fd03019dd20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 --2- 172.21.15.72:0/2180436655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0301077a0 0x7fd03019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.047+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 >> 172.21.15.72:0/2180436655 conn(0x7fd030076740 msgr2=0x7fd03010bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.048+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 shutdown_connections 2024-03-11T19:18:57.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.048+0000 7fd034c00640 1 -- 172.21.15.72:0/2180436655 wait complete. 2024-03-11T19:18:57.072 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:18:57.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 -- 172.21.15.72:0/4184982077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01089b0 msgr2=0x7f83f0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 --2- 172.21.15.72:0/4184982077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01089b0 0x7f83f0108e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009860 tx=0x7f83d802bb70 comp rx=0 tx=0).stop 2024-03-11T19:18:57.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 -- 172.21.15.72:0/4184982077 shutdown_connections 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 --2- 172.21.15.72:0/4184982077 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f0109370 0x7f83f010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 --2- 172.21.15.72:0/4184982077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01089b0 0x7f83f0108e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 --2- 172.21.15.72:0/4184982077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f01077c0 0x7f83f0107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 -- 172.21.15.72:0/4184982077 >> 172.21.15.72:0/4184982077 conn(0x7f83f0076790 msgr2=0x7f83f0078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 -- 172.21.15.72:0/4184982077 shutdown_connections 2024-03-11T19:18:57.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.272+0000 7f83f5c00640 1 -- 172.21.15.72:0/4184982077 wait complete. 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 Processor -- start 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 -- start start 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 0x7f83f019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f01089b0 0x7f83f019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 0x7f83f01a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01a5360 con 0x7f83f01089b0 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01a54d0 con 0x7f83f0109370 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.273+0000 7f83f5c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01a5690 con 0x7f83f01077c0 2024-03-11T19:18:57.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83e6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f01089b0 0x7f83f019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83eea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 0x7f83f019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83eea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 0x7f83f019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:52742/0 (socket says 172.21.15.72:52742) 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83eea00640 1 -- 172.21.15.72:0/3542495618 learned_addr learned my addr 172.21.15.72:0/3542495618 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 0x7f83f01a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 -- 172.21.15.72:0/3542495618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 msgr2=0x7f83f019d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 0x7f83f019d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 -- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f01089b0 msgr2=0x7f83f019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f01089b0 0x7f83f019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.274+0000 7f83ef400640 1 -- 172.21.15.72:0/3542495618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8009900 con 0x7f83f0109370 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83ef400640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 0x7f83f01a4e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83e800efc0 tx=0x7f83e800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8003970 con 0x7f83f0109370 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004020 con 0x7f83f0109370 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d8009490 con 0x7f83f0109370 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e800b960 con 0x7f83f0109370 2024-03-11T19:18:57.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.275+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f01a5c70 con 0x7f83f0109370 2024-03-11T19:18:57.281 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.276+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f010eae0 con 0x7f83f0109370 2024-03-11T19:18:57.281 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.278+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f83e800a040 con 0x7f83f0109370 2024-03-11T19:18:57.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.279+0000 7f83e7e00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 0x7f83d40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:57.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.279+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83e80cd200 con 0x7f83f0109370 2024-03-11T19:18:57.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.279+0000 7f83eea00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 0x7f83d40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:57.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.280+0000 7f83eea00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 0x7f83d40a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83e00058c0 tx=0x7f83e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:57.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.282+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83e8096810 con 0x7f83f0109370 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:57.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: Deploying daemon mds.l on smithi143 2024-03-11T19:18:57.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2708077175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:57.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:57 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2180436655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:57.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:57.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: Deploying daemon mds.l on smithi143 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2708077175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:57.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:57 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2180436655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:18:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:57.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:57.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:57.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: Deploying daemon mds.l on smithi143 2024-03-11T19:18:57.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2708077175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:18:57.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:57 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2180436655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:18:57.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.519+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83f002ce40 con 0x7f83f0109370 2024-03-11T19:18:57.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.520+0000 7f83e7e00640 1 -- 172.21.15.72:0/3542495618 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f83e8099ed0 con 0x7f83f0109370 2024-03-11T19:18:57.524 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:18:57.525 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-11T19:18:57.525 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 1 2024-03-11T19:18:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 msgr2=0x7f83d40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 0x7f83d40a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83e00058c0 tx=0x7f83e0005f90 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 msgr2=0x7f83f01a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 0x7f83f01a4e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83e800efc0 tx=0x7f83e800c800 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 shutdown_connections 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f83d40a1a30 0x7f83d40a3ef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83f0109370 0x7f83f01a4e20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83f01089b0 0x7f83f019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 --2- 172.21.15.72:0/3542495618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83f01077c0 0x7f83f019d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 >> 172.21.15.72:0/3542495618 conn(0x7f83f0076790 msgr2=0x7f83f010bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 shutdown_connections 2024-03-11T19:18:57.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:57.523+0000 7f83f5c00640 1 -- 172.21.15.72:0/3542495618 wait complete. 2024-03-11T19:18:58.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:58 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3542495618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:18:58.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:58 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3542495618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:18:58.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:58 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3542495618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:18:58.623 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 systemd[1]: Starting Ceph mds.l for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:18:58.921 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 podman[66166]: 2024-03-11 19:18:58.522748898 +0000 UTC m=+0.017934371 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:18:58.921 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 podman[66166]: 2024-03-11 19:18:58.673442053 +0000 UTC m=+0.168627559 container create 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True) 2024-03-11T19:18:58.921 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 podman[66166]: 2024-03-11 19:18:58.918087428 +0000 UTC m=+0.413272916 container init 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-11T19:18:59.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:59.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6832/239358886,v1:172.21.15.143:6833/239358886] up:boot 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: fsmap 1 up:standby 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:18:59 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:59.190 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 podman[66166]: 2024-03-11 19:18:58.92059611 +0000 UTC m=+0.415781571 container start 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-03-11T19:18:59.191 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 ceph-mds[66185]: -- 172.21.15.143:0/4194639957 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564e81f661a0 con 0x564e81f3d000 2024-03-11T19:18:59.191 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l[66181]: starting mds.l at 2024-03-11T19:18:59.191 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:59 smithi143 bash[66166]: 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 2024-03-11T19:18:59.191 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:18:59 smithi143 systemd[1]: Started Ceph mds.l for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6832/239358886,v1:172.21.15.143:6833/239358886] up:boot 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: fsmap 1 up:standby 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:59.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:59.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:18:59 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6832/239358886,v1:172.21.15.143:6833/239358886] up:boot 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: fsmap 1 up:standby 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:18:59.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:18:59.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:18:59.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:18:59 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:18:59.548 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:18:59.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 -- 172.21.15.72:0/3946004777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1075e0 msgr2=0x7f215c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 --2- 172.21.15.72:0/3946004777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1075e0 0x7f215c107a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f214c009900 tx=0x7f214c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 -- 172.21.15.72:0/3946004777 shutdown_connections 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 --2- 172.21.15.72:0/3946004777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c109190 0x7f215c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 --2- 172.21.15.72:0/3946004777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c1087d0 0x7f215c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 --2- 172.21.15.72:0/3946004777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1075e0 0x7f215c107a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 -- 172.21.15.72:0/3946004777 >> 172.21.15.72:0/3946004777 conn(0x7f215c076590 msgr2=0x7f215c0789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.767+0000 7f215b200640 1 -- 172.21.15.72:0/3946004777 shutdown_connections 2024-03-11T19:18:59.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.768+0000 7f215b200640 1 -- 172.21.15.72:0/3946004777 wait complete. 2024-03-11T19:18:59.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.768+0000 7f215b200640 1 Processor -- start 2024-03-11T19:18:59.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 -- start start 2024-03-11T19:18:59.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:59.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 0x7f215c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:59.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c109190 0x7f215c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c1a51b0 con 0x7f215c1087d0 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c1a5320 con 0x7f215c1075e0 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c1a54e0 con 0x7f215c109190 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f2151400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 0x7f215c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f2159e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f2159e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:49130/0 (socket says 172.21.15.72:49130) 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f2151400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 0x7f215c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52234/0 (socket says 172.21.15.72:52234) 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f2159e00640 1 -- 172.21.15.72:0/1902249514 learned_addr learned my addr 172.21.15.72:0/1902249514 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.769+0000 7f215a800640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c109190 0x7f215c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 -- 172.21.15.72:0/1902249514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c109190 msgr2=0x7f215c1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c109190 0x7f215c1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 -- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 msgr2=0x7f215c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 0x7f215c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:18:59.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 -- 172.21.15.72:0/1902249514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009560 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2159e00640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f214c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0044c0 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f214c0052d0 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c1a5760 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c003e80 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.770+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c1a5cc0 con 0x7f215c1075e0 2024-03-11T19:18:59.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.772+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c10e950 con 0x7f215c1075e0 2024-03-11T19:18:59.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.773+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f214c005440 con 0x7f215c1075e0 2024-03-11T19:18:59.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.775+0000 7f2153400640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 0x7f21400a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:18:59.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.775+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f214c0ef7e0 con 0x7f215c1075e0 2024-03-11T19:18:59.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.775+0000 7f2151400640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 0x7f21400a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:18:59.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.775+0000 7f2151400640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 0x7f21400a3f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f215c19ebd0 tx=0x7f2144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:18:59.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:18:59.779+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f214c0b8f10 con 0x7f215c1075e0 2024-03-11T19:19:00.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.012+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f215c113010 con 0x7f215c1075e0 2024-03-11T19:19:00.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.013+0000 7f2153400640 1 -- 172.21.15.72:0/1902249514 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f214c0bc5d0 con 0x7f215c1075e0 2024-03-11T19:19:00.017 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:00.017 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:19:00.017 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 2 2024-03-11T19:19:00.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 msgr2=0x7f21400a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:00.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 0x7f21400a3f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f215c19ebd0 tx=0x7f2144005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:00.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 msgr2=0x7f215c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f214c0040c0 comp rx=0 tx=0).stop 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 shutdown_connections 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21400a1ab0 0x7f21400a3f70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c109190 0x7f215c1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f215c1087d0 0x7f215c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 --2- 172.21.15.72:0/1902249514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f215c1075e0 0x7f215c19d6c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:00.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 >> 172.21.15.72:0/1902249514 conn(0x7f215c076590 msgr2=0x7f215c10bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:00.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 shutdown_connections 2024-03-11T19:19:00.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:00.016+0000 7f215b200640 1 -- 172.21.15.72:0/1902249514 wait complete. 2024-03-11T19:19:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:00 smithi106 ceph-mon[35889]: Deploying daemon mds.k on smithi106 2024-03-11T19:19:00.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1902249514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:00.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:00 smithi072 ceph-mon[31695]: Deploying daemon mds.k on smithi106 2024-03-11T19:19:00.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1902249514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:00 smithi143 ceph-mon[36013]: Deploying daemon mds.k on smithi106 2024-03-11T19:19:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1902249514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:01 smithi072 ceph-mon[31695]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:01 smithi143 ceph-mon[36013]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:01.592 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:01 smithi106 ceph-mon[35889]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:01.593 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 systemd[1]: Starting Ceph mds.k for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:01.854 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 podman[65329]: 2024-03-11 19:19:01.491685988 +0000 UTC m=+0.017287541 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:01.854 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 podman[65329]: 2024-03-11 19:19:01.624179829 +0000 UTC m=+0.149781369 container create faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-11T19:19:01.855 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 podman[65329]: 2024-03-11 19:19:01.851804923 +0000 UTC m=+0.377406486 container init faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T19:19:02.040 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:02.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 podman[65329]: 2024-03-11 19:19:01.85455739 +0000 UTC m=+0.380158938 container start faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-11T19:19:02.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 ceph-mds[65348]: -- 172.21.15.106:0/435484677 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562fa374e1a0 con 0x562fa3724800 2024-03-11T19:19:02.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k[65344]: starting mds.k at 2024-03-11T19:19:02.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 bash[65329]: faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f 2024-03-11T19:19:02.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:19:01 smithi106 systemd[1]: Started Ceph mds.k for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 -- 172.21.15.72:0/730544419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1033c0 msgr2=0x7fb87c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 --2- 172.21.15.72:0/730544419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1033c0 0x7fb87c1037e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87002bc50 comp rx=0 tx=0).stop 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 -- 172.21.15.72:0/730544419 shutdown_connections 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 --2- 172.21.15.72:0/730544419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c104f70 0x7fb87c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 --2- 172.21.15.72:0/730544419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c1045b0 0x7fb87c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 --2- 172.21.15.72:0/730544419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1033c0 0x7fb87c1037e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 -- 172.21.15.72:0/730544419 >> 172.21.15.72:0/730544419 conn(0x7fb87c0fe970 msgr2=0x7fb87c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:02.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 -- 172.21.15.72:0/730544419 shutdown_connections 2024-03-11T19:19:02.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.233+0000 7fb883800640 1 -- 172.21.15.72:0/730544419 wait complete. 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 Processor -- start 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 -- start start 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c1033c0 0x7fb87c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c104f70 0x7fb87c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1a0f80 con 0x7fb87c1033c0 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1a10f0 con 0x7fb87c1045b0 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb883800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1a12b0 con 0x7fb87c104f70 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb87be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb880e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c1033c0 0x7fb87c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:02.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb87be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52626/0 (socket says 172.21.15.72:52626) 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.234+0000 7fb87be00640 1 -- 172.21.15.72:0/1111975287 learned_addr learned my addr 172.21.15.72:0/1111975287 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb881800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c104f70 0x7fb87c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 -- 172.21.15.72:0/1111975287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c104f70 msgr2=0x7fb87c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c104f70 0x7fb87c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 -- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c1033c0 msgr2=0x7fb87c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c1033c0 0x7fb87c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 -- 172.21.15.72:0/1111975287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009560 con 0x7fb87c1045b0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb87be00640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb86800e9f0 tx=0x7fb86800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868009720 con 0x7fb87c1045b0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb86800cd80 con 0x7fb87c1045b0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868010690 con 0x7fb87c1045b0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.235+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c1a1590 con 0x7fb87c1045b0 2024-03-11T19:19:02.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.236+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c1a1a50 con 0x7fb87c1045b0 2024-03-11T19:19:02.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.237+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c10a6e0 con 0x7fb87c1045b0 2024-03-11T19:19:02.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.238+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb8680046c0 con 0x7fb87c1045b0 2024-03-11T19:19:02.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.240+0000 7fb879600640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 0x7fb8640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:02.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.240+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb868014070 con 0x7fb87c1045b0 2024-03-11T19:19:02.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.241+0000 7fb880e00640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 0x7fb8640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:02.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.241+0000 7fb880e00640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 0x7fb8640a3ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:02.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.243+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb868096a30 con 0x7fb87c1045b0 2024-03-11T19:19:02.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.480+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb87c10ef80 con 0x7fb87c1045b0 2024-03-11T19:19:02.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.481+0000 7fb879600640 1 -- 172.21.15.72:0/1111975287 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fb86809a0f0 con 0x7fb87c1045b0 2024-03-11T19:19:02.485 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:02.485 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:19:02.485 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 msgr2=0x7fb8640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 0x7fb8640a3ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870005fd0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 msgr2=0x7fb87c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb86800e9f0 tx=0x7fb86800edf0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 shutdown_connections 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8640a1a30 0x7fb8640a3ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87c104f70 0x7fb87c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1045b0 0x7fb87c199a20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 --2- 172.21.15.72:0/1111975287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87c1033c0 0x7fb87c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 >> 172.21.15.72:0/1111975287 conn(0x7fb87c0fe970 msgr2=0x7fb87c1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:02.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 shutdown_connections 2024-03-11T19:19:02.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:02.484+0000 7fb883800640 1 -- 172.21.15.72:0/1111975287 wait complete. 2024-03-11T19:19:03.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:03.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: Deploying daemon mds.j on smithi072 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6832/4123124925,v1:172.21.15.106:6833/4123124925] up:boot 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: fsmap 2 up:standby 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:19:03.234 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:03 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1111975287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: Deploying daemon mds.j on smithi072 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6832/4123124925,v1:172.21.15.106:6833/4123124925] up:boot 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: fsmap 2 up:standby 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:19:03.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:03 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1111975287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: Deploying daemon mds.j on smithi072 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6832/4123124925,v1:172.21.15.106:6833/4123124925] up:boot 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: fsmap 2 up:standby 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:19:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:03 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1111975287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:04.508 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:04.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.882+0000 7f5288600640 1 -- 172.21.15.72:0/1954982844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f528010d770 msgr2=0x7f528010fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:04.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.882+0000 7f5288600640 1 --2- 172.21.15.72:0/1954982844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f528010d770 0x7f528010fb60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5278009c10 tx=0x7f527802bf30 comp rx=0 tx=0).stop 2024-03-11T19:19:04.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 -- 172.21.15.72:0/1954982844 shutdown_connections 2024-03-11T19:19:04.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 --2- 172.21.15.72:0/1954982844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f528010d770 0x7f528010fb60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:04.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 --2- 172.21.15.72:0/1954982844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 0x7f5280074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:04.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 --2- 172.21.15.72:0/1954982844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52800751c0 0x7f5280073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:04.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 -- 172.21.15.72:0/1954982844 >> 172.21.15.72:0/1954982844 conn(0x7f52800fc730 msgr2=0x7f52800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:04.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 -- 172.21.15.72:0/1954982844 shutdown_connections 2024-03-11T19:19:04.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.883+0000 7f5288600640 1 -- 172.21.15.72:0/1954982844 wait complete. 2024-03-11T19:19:04.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.885+0000 7f5288600640 1 Processor -- start 2024-03-11T19:19:04.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.885+0000 7f5288600640 1 -- start start 2024-03-11T19:19:04.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 0x7f5280109da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:04.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:04.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f528010d770 0x7f52801ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:04.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528010d330 con 0x7f52800751c0 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528010d4a0 con 0x7f5280073be0 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5288600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52801adf40 con 0x7f528010d770 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5286600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f528010d770 0x7f52801ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5285200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.886+0000 7f5285200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51244/0 (socket says 172.21.15.72:51244) 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 -- 172.21.15.72:0/938762023 learned_addr learned my addr 172.21.15.72:0/938762023 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:04.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285c00640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 0x7f5280109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 -- 172.21.15.72:0/938762023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f528010d770 msgr2=0x7f52801ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f528010d770 0x7f52801ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 -- 172.21.15.72:0/938762023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 msgr2=0x7f5280109da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 0x7f5280109da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 -- 172.21.15.72:0/938762023 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009560 con 0x7f52800751c0 2024-03-11T19:19:04.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.887+0000 7f5285200640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e9c0 tx=0x7f526c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:04.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.888+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00cde0 con 0x7f52800751c0 2024-03-11T19:19:04.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.888+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5278009840 con 0x7f52800751c0 2024-03-11T19:19:04.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.888+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f526c0035f0 con 0x7f52800751c0 2024-03-11T19:19:04.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.888+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c010650 con 0x7f52800751c0 2024-03-11T19:19:04.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.889+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52801ae4e0 con 0x7f52800751c0 2024-03-11T19:19:04.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.891+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5280103940 con 0x7f52800751c0 2024-03-11T19:19:04.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.894+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f526c003ad0 con 0x7f52800751c0 2024-03-11T19:19:04.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.896+0000 7f527e000640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 0x7f52680a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:04.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.896+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f526c014070 con 0x7f52800751c0 2024-03-11T19:19:04.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.896+0000 7f5285c00640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 0x7f52680a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:04.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.897+0000 7f5285c00640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 0x7f52680a3f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f527400bbe0 tx=0x7f527400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:04.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:04.901+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f526c0969b0 con 0x7f52800751c0 2024-03-11T19:19:05.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.139+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5280113370 con 0x7f52800751c0 2024-03-11T19:19:05.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.139+0000 7f527e000640 1 -- 172.21.15.72:0/938762023 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f526c003760 con 0x7f52800751c0 2024-03-11T19:19:05.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:05.144 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:19:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2024-03-11T19:19:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 msgr2=0x7f52680a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 0x7f52680a3f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f527400bbe0 tx=0x7f527400b390 comp rx=0 tx=0).stop 2024-03-11T19:19:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 msgr2=0x7f528010a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e9c0 tx=0x7f526c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 shutdown_connections 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52680a1a80 0x7f52680a3f40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f528010d770 0x7f52801ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52800751c0 0x7f528010a2e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 --2- 172.21.15.72:0/938762023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5280073be0 0x7f5280109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 >> 172.21.15.72:0/938762023 conn(0x7f52800fc730 msgr2=0x7f5280100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 shutdown_connections 2024-03-11T19:19:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:05.142+0000 7f5288600640 1 -- 172.21.15.72:0/938762023 wait complete. 2024-03-11T19:19:05.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:05 smithi106 ceph-mon[35889]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:05 smithi072 ceph-mon[31695]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:05 smithi143 ceph-mon[36013]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:05.830 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 podman[72137]: 2024-03-11 19:19:05.510101844 +0000 UTC m=+0.184357207 container create 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-11T19:19:05.830 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 podman[72137]: 2024-03-11 19:19:05.824734115 +0000 UTC m=+0.498989488 container init 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-11T19:19:06.113 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 podman[72137]: 2024-03-11 19:19:05.835765661 +0000 UTC m=+0.510021010 container start 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-11T19:19:06.113 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 ceph-mds[72156]: -- 172.21.15.72:0/3372848271 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560ddb7321a0 con 0x560ddb709000 2024-03-11T19:19:06.113 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j[72152]: starting mds.j at 2024-03-11T19:19:06.113 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 bash[72137]: 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b 2024-03-11T19:19:06.113 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:19:05 smithi072 systemd[1]: Started Ceph mds.j for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/938762023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:06.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:06 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/938762023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:06 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/938762023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:06.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:06 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:07.166 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:07.311 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:07.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:07.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: Deploying daemon mds.d on smithi072 2024-03-11T19:19:07.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6834/1028801800,v1:172.21.15.72:6835/1028801800] up:boot 2024-03-11T19:19:07.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: fsmap 3 up:standby 2024-03-11T19:19:07.312 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:07 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: Deploying daemon mds.d on smithi072 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6834/1028801800,v1:172.21.15.72:6835/1028801800] up:boot 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: fsmap 3 up:standby 2024-03-11T19:19:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:07 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 -- 172.21.15.72:0/642154902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120073b60 msgr2=0x7f9120073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 --2- 172.21.15.72:0/642154902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120073b60 0x7f9120073fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9108009860 tx=0x7f910802bc00 comp rx=0 tx=0).stop 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 -- 172.21.15.72:0/642154902 shutdown_connections 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 --2- 172.21.15.72:0/642154902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120109050 0x7f912010b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 --2- 172.21.15.72:0/642154902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120073b60 0x7f9120073fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 --2- 172.21.15.72:0/642154902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.393+0000 7f9127200640 1 -- 172.21.15.72:0/642154902 >> 172.21.15.72:0/642154902 conn(0x7f91200fc730 msgr2=0x7f91200feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.394+0000 7f9127200640 1 -- 172.21.15.72:0/642154902 shutdown_connections 2024-03-11T19:19:07.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.394+0000 7f9127200640 1 -- 172.21.15.72:0/642154902 wait complete. 2024-03-11T19:19:07.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.394+0000 7f9127200640 1 Processor -- start 2024-03-11T19:19:07.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 -- start start 2024-03-11T19:19:07.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120073b60 0x7f9120199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:07.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120109050 0x7f91201a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a0f40 con 0x7f9120075140 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a10b0 con 0x7f9120073b60 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9127200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91201a1270 con 0x7f9120109050 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f911f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f911f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51262/0 (socket says 172.21.15.72:51262) 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f911f400640 1 -- 172.21.15.72:0/2391479337 learned_addr learned my addr 172.21.15.72:0/2391479337 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f911fe00640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120073b60 0x7f9120199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.395+0000 7f9125200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120109050 0x7f91201a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 -- 172.21.15.72:0/2391479337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120109050 msgr2=0x7f91201a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120109050 0x7f91201a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 -- 172.21.15.72:0/2391479337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120073b60 msgr2=0x7f9120199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:07.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120073b60 0x7f9120199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 -- 172.21.15.72:0/2391479337 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009900 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911f400640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9108034b70 tx=0x7f91080346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108005220 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910802ca80 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9108009490 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108004d60 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.396+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a1810 con 0x7f9120075140 2024-03-11T19:19:07.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.398+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9120107e10 con 0x7f9120075140 2024-03-11T19:19:07.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.399+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f910803b040 con 0x7f9120075140 2024-03-11T19:19:07.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.401+0000 7f911cc00640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 0x7f91040a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:07.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.401+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91080eed60 con 0x7f9120075140 2024-03-11T19:19:07.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.401+0000 7f911fe00640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 0x7f91040a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:07.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.401+0000 7f911fe00640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 0x7f91040a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9110009bd0 tx=0x7f9110005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:07.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.405+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91080b8420 con 0x7f9120075140 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: Deploying daemon mds.d on smithi072 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6834/1028801800,v1:172.21.15.72:6835/1028801800] up:boot 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: fsmap 3 up:standby 2024-03-11T19:19:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:07 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:19:07.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.658+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f912002cdd0 con 0x7f9120075140 2024-03-11T19:19:07.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.658+0000 7f911cc00640 1 -- 172.21.15.72:0/2391479337 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f910804ab70 con 0x7f9120075140 2024-03-11T19:19:07.663 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:07.663 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:19:07.663 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 4 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 msgr2=0x7f91040a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 0x7f91040a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9110009bd0 tx=0x7f9110005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 msgr2=0x7f9120199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9108034b70 tx=0x7f91080346f0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 shutdown_connections 2024-03-11T19:19:07.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91040a1a30 0x7f91040a3ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9120109050 0x7f91201a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9120075140 0x7f9120199990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 --2- 172.21.15.72:0/2391479337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9120073b60 0x7f9120199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 >> 172.21.15.72:0/2391479337 conn(0x7f91200fc730 msgr2=0x7f9120104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 shutdown_connections 2024-03-11T19:19:07.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:07.661+0000 7f9127200640 1 -- 172.21.15.72:0/2391479337 wait complete. 2024-03-11T19:19:08.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2391479337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:08.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2391479337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:08.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2391479337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:09.269 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:09 smithi072 ceph-mon[31695]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:09.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:09 smithi106 ceph-mon[35889]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:09 smithi143 ceph-mon[36013]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:09.669 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:09 smithi072 systemd[1]: Starting Ceph mds.d for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:09.684 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.902+0000 7f341f800640 1 -- 172.21.15.72:0/3782964239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418076d80 msgr2=0x7f34180771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.902+0000 7f341f800640 1 --2- 172.21.15.72:0/3782964239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418076d80 0x7f34180771e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3404009900 tx=0x7f340402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 -- 172.21.15.72:0/3782964239 shutdown_connections 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 --2- 172.21.15.72:0/3782964239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418077720 0x7f3418077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 --2- 172.21.15.72:0/3782964239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418076d80 0x7f34180771e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 --2- 172.21.15.72:0/3782964239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 0x7f3418075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 -- 172.21.15.72:0/3782964239 >> 172.21.15.72:0/3782964239 conn(0x7f34180fe970 msgr2=0x7f3418100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 -- 172.21.15.72:0/3782964239 shutdown_connections 2024-03-11T19:19:09.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.903+0000 7f341f800640 1 -- 172.21.15.72:0/3782964239 wait complete. 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 Processor -- start 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 -- start start 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 0x7f341819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418076d80 0x7f341819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:09.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181a5360 con 0x7f3418077720 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181a54d0 con 0x7f3418076d80 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.904+0000 7f341f800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181a5690 con 0x7f3418075b40 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51268/0 (socket says 172.21.15.72:51268) 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 -- 172.21.15.72:0/1524692168 learned_addr learned my addr 172.21.15.72:0/1524692168 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341ce00640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 0x7f341819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f3417e00640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418076d80 0x7f341819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 -- 172.21.15.72:0/1524692168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 msgr2=0x7f341819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 0x7f341819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 -- 172.21.15.72:0/1524692168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418076d80 msgr2=0x7f341819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418076d80 0x7f341819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 -- 172.21.15.72:0/1524692168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404009560 con 0x7f3418077720 2024-03-11T19:19:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.905+0000 7f341d800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f341000eb10 tx=0x7f341000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.906+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410004b20 con 0x7f3418077720 2024-03-11T19:19:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.906+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3410004c80 con 0x7f3418077720 2024-03-11T19:19:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.906+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410010650 con 0x7f3418077720 2024-03-11T19:19:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.906+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181a5970 con 0x7f3418077720 2024-03-11T19:19:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.906+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181a5e50 con 0x7f3418077720 2024-03-11T19:19:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.907+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341810fe70 con 0x7f3418077720 2024-03-11T19:19:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.909+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3410005020 con 0x7f3418077720 2024-03-11T19:19:09.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.910+0000 7f3415600640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 0x7f34000a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:09.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.910+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3410014070 con 0x7f3418077720 2024-03-11T19:19:09.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.911+0000 7f341ce00640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 0x7f34000a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:09.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.911+0000 7f341ce00640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 0x7f34000a3f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f340c009bd0 tx=0x7f340c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:09.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:09.914+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3410096b60 con 0x7f3418077720 2024-03-11T19:19:09.930 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:09 smithi072 podman[72482]: 2024-03-11 19:19:09.570713506 +0000 UTC m=+0.034398447 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:09.930 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:09 smithi072 podman[72482]: 2024-03-11 19:19:09.71047624 +0000 UTC m=+0.174161189 container create dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:19:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.154+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f341802ce40 con 0x7f3418077720 2024-03-11T19:19:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.154+0000 7f3415600640 1 -- 172.21.15.72:0/1524692168 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f341009a220 con 0x7f3418077720 2024-03-11T19:19:10.158 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:10.159 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-03-11T19:19:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 5 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 msgr2=0x7f34000a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 0x7f34000a3f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f340c009bd0 tx=0x7f340c005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 msgr2=0x7f34181a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f341000eb10 tx=0x7f341000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 shutdown_connections 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34000a1ab0 0x7f34000a3f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3418077720 0x7f34181a4e20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3418076d80 0x7f341819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 --2- 172.21.15.72:0/1524692168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418075b40 0x7f341819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 >> 172.21.15.72:0/1524692168 conn(0x7f34180fe970 msgr2=0x7f34181000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 shutdown_connections 2024-03-11T19:19:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:10.157+0000 7f341f800640 1 -- 172.21.15.72:0/1524692168 wait complete. 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 podman[72482]: 2024-03-11 19:19:10.053457239 +0000 UTC m=+0.517142221 container init dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS) 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 podman[72482]: 2024-03-11 19:19:10.062804824 +0000 UTC m=+0.526489770 container start dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 ceph-mds[72540]: -- 172.21.15.72:0/2102536633 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e952421a0 con 0x560e95219000 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d[72517]: starting mds.d at 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 bash[72482]: dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e 2024-03-11T19:19:10.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:19:10 smithi072 systemd[1]: Started Ceph mds.d for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:11.242 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:11.242 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6836/1584863054,v1:172.21.15.72:6837/1584863054] up:boot 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: fsmap 4 up:standby 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1524692168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:11.243 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:11 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:11.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6836/1584863054,v1:172.21.15.72:6837/1584863054] up:boot 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: fsmap 4 up:standby 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1524692168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:11.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:11 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:11.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6836/1584863054,v1:172.21.15.72:6837/1584863054] up:boot 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: fsmap 4 up:standby 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1524692168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:11.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:11.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:11.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:11 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:12.180 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:12.354 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:12 smithi106 ceph-mon[35889]: Deploying daemon mds.e on smithi106 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.397+0000 7f6a19c00640 1 -- 172.21.15.72:0/3200652876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a141089b0 msgr2=0x7f6a14108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.397+0000 7f6a19c00640 1 --2- 172.21.15.72:0/3200652876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a141089b0 0x7f6a14108e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009860 tx=0x7f69fc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 -- 172.21.15.72:0/3200652876 shutdown_connections 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 --2- 172.21.15.72:0/3200652876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a14109370 0x7f6a1410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 --2- 172.21.15.72:0/3200652876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a141089b0 0x7f6a14108e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 --2- 172.21.15.72:0/3200652876 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a14107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 -- 172.21.15.72:0/3200652876 >> 172.21.15.72:0/3200652876 conn(0x7f6a14076790 msgr2=0x7f6a14078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 -- 172.21.15.72:0/3200652876 shutdown_connections 2024-03-11T19:19:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.398+0000 7f6a19c00640 1 -- 172.21.15.72:0/3200652876 wait complete. 2024-03-11T19:19:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.399+0000 7f6a19c00640 1 Processor -- start 2024-03-11T19:19:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.399+0000 7f6a19c00640 1 -- start start 2024-03-11T19:19:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.399+0000 7f6a19c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.399+0000 7f6a19c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a141089b0 0x7f6a1419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a19c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 0x7f6a141a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a19c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141a52f0 con 0x7f6a141077c0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a19c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141a5460 con 0x7f6a14109370 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a19c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141a5620 con 0x7f6a141089b0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43206/0 (socket says 172.21.15.72:43206) 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a13400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 0x7f6a141a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 -- 172.21.15.72:0/2728077115 learned_addr learned my addr 172.21.15.72:0/2728077115 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a0a000640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a141089b0 0x7f6a1419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 -- 172.21.15.72:0/2728077115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a141089b0 msgr2=0x7f6a1419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a141089b0 0x7f6a1419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 -- 172.21.15.72:0/2728077115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 msgr2=0x7f6a141a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 0x7f6a141a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a12a00640 1 -- 172.21.15.72:0/2728077115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc009490 con 0x7f6a141077c0 2024-03-11T19:19:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.400+0000 7f6a13400640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 0x7f6a141a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a12a00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c3a0 tx=0x7f6a0400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04003970 con 0x7f6a141077c0 2024-03-11T19:19:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a04004020 con 0x7f6a141077c0 2024-03-11T19:19:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400b940 con 0x7f6a141077c0 2024-03-11T19:19:12.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a141a5900 con 0x7f6a141077c0 2024-03-11T19:19:12.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.401+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a141a5d90 con 0x7f6a141077c0 2024-03-11T19:19:12.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.402+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1410eae0 con 0x7f6a141077c0 2024-03-11T19:19:12.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.404+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6a0400a040 con 0x7f6a141077c0 2024-03-11T19:19:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.406+0000 7f6a0be00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 0x7f69f80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.406+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6a040cd830 con 0x7f6a141077c0 2024-03-11T19:19:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.406+0000 7f6a0a000640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 0x7f69f80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:12.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.407+0000 7f6a0a000640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 0x7f69f80a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69fc02c180 tx=0x7f69fc034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:12.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.408+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a04096e40 con 0x7f6a141077c0 2024-03-11T19:19:12.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:12 smithi072 ceph-mon[31695]: Deploying daemon mds.e on smithi106 2024-03-11T19:19:12.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:12 smithi143 ceph-mon[36013]: Deploying daemon mds.e on smithi106 2024-03-11T19:19:12.645 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:12 smithi106 systemd[1]: Starting Ceph mds.e for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:12.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.644+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a14113470 con 0x7f6a141077c0 2024-03-11T19:19:12.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.645+0000 7f6a0be00640 1 -- 172.21.15.72:0/2728077115 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f6a14113470 con 0x7f6a141077c0 2024-03-11T19:19:12.649 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:12.649 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-03-11T19:19:12.649 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 5 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.647+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 msgr2=0x7f69f80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.647+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 0x7f69f80a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69fc02c180 tx=0x7f69fc034c00 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 msgr2=0x7f6a1419d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c3a0 tx=0x7f6a0400c860 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 shutdown_connections 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69f80a1a30 0x7f69f80a3ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a14109370 0x7f6a141a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a141089b0 0x7f6a1419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 --2- 172.21.15.72:0/2728077115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a141077c0 0x7f6a1419d800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 >> 172.21.15.72:0/2728077115 conn(0x7f6a14076790 msgr2=0x7f6a1410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:12.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 shutdown_connections 2024-03-11T19:19:12.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:12.648+0000 7f6a19c00640 1 -- 172.21.15.72:0/2728077115 wait complete. 2024-03-11T19:19:13.010 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:12 smithi106 podman[65646]: 2024-03-11 19:19:12.648445921 +0000 UTC m=+0.017939138 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:13.010 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:12 smithi106 podman[65646]: 2024-03-11 19:19:12.762521035 +0000 UTC m=+0.132014298 container create ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T19:19:13.010 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 podman[65646]: 2024-03-11 19:19:13.00694864 +0000 UTC m=+0.376441843 container init ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:19:13.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:13 smithi106 ceph-mon[35889]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:13.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:13 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2728077115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:13.396 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 podman[65646]: 2024-03-11 19:19:13.009573824 +0000 UTC m=+0.379067030 container start ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:19:13.396 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 ceph-mds[65665]: -- 172.21.15.106:0/336801064 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556bb331e1a0 con 0x556bb32f4800 2024-03-11T19:19:13.396 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e[65661]: starting mds.e at 2024-03-11T19:19:13.396 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 bash[65646]: ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 2024-03-11T19:19:13.397 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:19:13 smithi106 systemd[1]: Started Ceph mds.e for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:13.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:13 smithi072 ceph-mon[31695]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:13.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:13 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2728077115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:13.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:13 smithi143 ceph-mon[36013]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:13.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:13 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2728077115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:14.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6834/456413378,v1:172.21.15.106:6835/456413378] up:boot 2024-03-11T19:19:14.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: fsmap 5 up:standby 2024-03-11T19:19:14.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:14.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:14 smithi072 ceph-mon[31695]: Deploying daemon mds.f on smithi143 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6834/456413378,v1:172.21.15.106:6835/456413378] up:boot 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: fsmap 5 up:standby 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:14.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:14.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:14.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:14 smithi143 ceph-mon[36013]: Deploying daemon mds.f on smithi143 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6834/456413378,v1:172.21.15.106:6835/456413378] up:boot 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: fsmap 5 up:standby 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:14 smithi106 ceph-mon[35889]: Deploying daemon mds.f on smithi143 2024-03-11T19:19:14.672 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2060655744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41031d0 msgr2=0x7fb8c41035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2060655744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41031d0 0x7fb8c41035f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2060655744 shutdown_connections 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2060655744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2060655744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41043c0 0x7fb8c4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2060655744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41031d0 0x7fb8c41035f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2060655744 >> 172.21.15.72:0/2060655744 conn(0x7fb8c40fe760 msgr2=0x7fb8c4100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2060655744 shutdown_connections 2024-03-11T19:19:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.889+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2060655744 wait complete. 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 Processor -- start 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 -- start start 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 0x7fb8c41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41043c0 0x7fb8c4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a0dc0 con 0x7fb8c4104d80 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a0f30 con 0x7fb8c41031d0 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a10f0 con 0x7fb8c41043c0 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 0x7fb8c41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c41a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43222/0 (socket says 172.21.15.72:43222) 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.891+0000 7fb8c9400640 1 -- 172.21.15.72:0/2242157775 learned_addr learned my addr 172.21.15.72:0/2242157775 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8bbe00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41043c0 0x7fb8c4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 -- 172.21.15.72:0/2242157775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41043c0 msgr2=0x7fb8c4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41043c0 0x7fb8c4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 -- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 msgr2=0x7fb8c41a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c41a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 -- 172.21.15.72:0/2242157775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009560 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.892+0000 7fb8c8a00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 0x7fb8c41992d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4036040 tx=0x7fb8b4005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.893+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4004490 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.893+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8b40052d0 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.893+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41a1370 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.893+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4003c80 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.893+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a1850 con 0x7fb8c41031d0 2024-03-11T19:19:14.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.894+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c410a490 con 0x7fb8c41031d0 2024-03-11T19:19:14.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.896+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb8b4004df0 con 0x7fb8c41031d0 2024-03-11T19:19:14.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.897+0000 7fb8c2000640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 0x7fb8a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:14.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.897+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8b40efd30 con 0x7fb8c41031d0 2024-03-11T19:19:14.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.897+0000 7fb8bbe00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 0x7fb8a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:14.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.898+0000 7fb8bbe00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 0x7fb8a80a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb8c419a7e0 tx=0x7fb8ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:14.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:14.899+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8b40b9340 con 0x7fb8c41031d0 2024-03-11T19:19:15.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.136+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8c402cdd0 con 0x7fb8c41031d0 2024-03-11T19:19:15.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.137+0000 7fb8c2000640 1 -- 172.21.15.72:0/2242157775 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fb8b40bca00 con 0x7fb8c41031d0 2024-03-11T19:19:15.141 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:15.141 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-11T19:19:15.141 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 6 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.139+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 msgr2=0x7fb8a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.139+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 0x7fb8a80a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb8c419a7e0 tx=0x7fb8ac005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.139+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 msgr2=0x7fb8c41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.139+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 0x7fb8c41992d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4036040 tx=0x7fb8b4005e40 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 shutdown_connections 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb8a80a1a30 0x7fb8a80a3ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8c4104d80 0x7fb8c41a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8c41043c0 0x7fb8c4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 --2- 172.21.15.72:0/2242157775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8c41031d0 0x7fb8c41992d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 >> 172.21.15.72:0/2242157775 conn(0x7fb8c40fe760 msgr2=0x7fb8c4107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 shutdown_connections 2024-03-11T19:19:15.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:15.140+0000 7fb8c9e00640 1 -- 172.21.15.72:0/2242157775 wait complete. 2024-03-11T19:19:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:15 smithi072 ceph-mon[31695]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:15 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2242157775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:15.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:15 smithi143 ceph-mon[36013]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:15.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:15 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2242157775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:15.497 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:15 smithi143 systemd[1]: Starting Ceph mds.f for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:15 smithi106 ceph-mon[35889]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:15 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2242157775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:15.795 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:15 smithi143 podman[66490]: 2024-03-11 19:19:15.61115966 +0000 UTC m=+0.017532760 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:15.795 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:15 smithi143 podman[66490]: 2024-03-11 19:19:15.791547137 +0000 UTC m=+0.197920218 container create bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:19:16.056 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 podman[66490]: 2024-03-11 19:19:16.028262627 +0000 UTC m=+0.434635713 container init bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T19:19:16.056 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 podman[66490]: 2024-03-11 19:19:16.030923846 +0000 UTC m=+0.437296943 container start bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-11T19:19:16.439 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 ceph-mds[66509]: -- 172.21.15.143:0/1604164534 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f920f1c1a0 con 0x55f920ef2c00 2024-03-11T19:19:16.439 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: starting mds.f at 2024-03-11T19:19:16.439 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 bash[66490]: bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 2024-03-11T19:19:16.439 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:19:16 smithi143 systemd[1]: Started Ceph mds.f for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:17.163 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:17.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 -- 172.21.15.72:0/1231581533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 msgr2=0x7f38e40771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:17.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 --2- 172.21.15.72:0/1231581533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e40771e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009900 tx=0x7f38cc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:17.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 -- 172.21.15.72:0/1231581533 shutdown_connections 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 --2- 172.21.15.72:0/1231581533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4077720 0x7f38e4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 --2- 172.21.15.72:0/1231581533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e40771e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 --2- 172.21.15.72:0/1231581533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4075b40 0x7f38e4075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 -- 172.21.15.72:0/1231581533 >> 172.21.15.72:0/1231581533 conn(0x7f38e40fe930 msgr2=0x7f38e4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 -- 172.21.15.72:0/1231581533 shutdown_connections 2024-03-11T19:19:17.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.377+0000 7f38eac00640 1 -- 172.21.15.72:0/1231581533 wait complete. 2024-03-11T19:19:17.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 Processor -- start 2024-03-11T19:19:17.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 -- start start 2024-03-11T19:19:17.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:17.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4077720 0x7f38e41a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.378+0000 7f38eac00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a5360 con 0x7f38e4075b40 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38eac00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a54d0 con 0x7f38e4077720 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38eac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41a5690 con 0x7f38e4076d80 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43244/0 (socket says 172.21.15.72:43244) 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3e00640 1 -- 172.21.15.72:0/1972935853 learned_addr learned my addr 172.21.15.72:0/1972935853 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e8c00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4077720 0x7f38e41a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 -- 172.21.15.72:0/1972935853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4077720 msgr2=0x7f38e41a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4077720 0x7f38e41a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 -- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 msgr2=0x7f38e419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 -- 172.21.15.72:0/1972935853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009560 con 0x7f38e4076d80 2024-03-11T19:19:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3e00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.379+0000 7f38e3400640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e419ddb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38cc000c00 tx=0x7f38cc005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.380+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0043d0 con 0x7f38e4076d80 2024-03-11T19:19:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.380+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38cc0052d0 con 0x7f38e4076d80 2024-03-11T19:19:17.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.380+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e41a5910 con 0x7f38e4076d80 2024-03-11T19:19:17.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.380+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc003d40 con 0x7f38e4076d80 2024-03-11T19:19:17.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.380+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41a5df0 con 0x7f38e4076d80 2024-03-11T19:19:17.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.381+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e410feb0 con 0x7f38e4076d80 2024-03-11T19:19:17.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.383+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f38cc005440 con 0x7f38e4076d80 2024-03-11T19:19:17.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.384+0000 7f38e0c00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 0x7f38c80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:17.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.384+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38cc0ef310 con 0x7f38e4076d80 2024-03-11T19:19:17.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.384+0000 7f38e3e00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 0x7f38c80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:17.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.385+0000 7f38e3e00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 0x7f38c80a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38d40049d0 tx=0x7f38d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:17.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.386+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38cc0b8920 con 0x7f38e4076d80 2024-03-11T19:19:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:17.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: Deploying daemon mds.c on smithi143 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6834/2743792791,v1:172.21.15.143:6835/2743792791] up:boot 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: fsmap 6 up:standby 2024-03-11T19:19:17.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:17 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:19:17.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.620+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38e41131c0 con 0x7f38e4076d80 2024-03-11T19:19:17.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.621+0000 7f38e0c00640 1 -- 172.21.15.72:0/1972935853 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f38cc0bbfe0 con 0x7f38e4076d80 2024-03-11T19:19:17.625 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:17.625 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-11T19:19:17.626 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 7 2024-03-11T19:19:17.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 msgr2=0x7f38c80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:17.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 0x7f38c80a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38d40049d0 tx=0x7f38d4009500 comp rx=0 tx=0).stop 2024-03-11T19:19:17.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 msgr2=0x7f38e419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e419ddb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38cc000c00 tx=0x7f38cc005e50 comp rx=0 tx=0).stop 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 shutdown_connections 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f38c80a1a30 0x7f38c80a3ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38e4077720 0x7f38e41a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4076d80 0x7f38e419ddb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 --2- 172.21.15.72:0/1972935853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38e4075b40 0x7f38e419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.623+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 >> 172.21.15.72:0/1972935853 conn(0x7f38e40fe930 msgr2=0x7f38e4100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.624+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 shutdown_connections 2024-03-11T19:19:17.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:17.624+0000 7f38eac00640 1 -- 172.21.15.72:0/1972935853 wait complete. 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:17.643 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:17.644 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:17.644 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: Deploying daemon mds.c on smithi143 2024-03-11T19:19:17.644 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6834/2743792791,v1:172.21.15.143:6835/2743792791] up:boot 2024-03-11T19:19:17.644 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: fsmap 6 up:standby 2024-03-11T19:19:17.644 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:17 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:17.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: Deploying daemon mds.c on smithi143 2024-03-11T19:19:17.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6834/2743792791,v1:172.21.15.143:6835/2743792791] up:boot 2024-03-11T19:19:17.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: fsmap 6 up:standby 2024-03-11T19:19:17.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:17 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:19:18.319 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:18 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1972935853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:18.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:18 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1972935853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:18.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:18 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1972935853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:18.689 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:18 smithi143 systemd[1]: Starting Ceph mds.c for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:19.006 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:18 smithi143 podman[66802]: 2024-03-11 19:19:18.618230576 +0000 UTC m=+0.017218446 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:19.006 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:18 smithi143 podman[66802]: 2024-03-11 19:19:18.73310398 +0000 UTC m=+0.132091830 container create 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-11T19:19:19.006 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 podman[66802]: 2024-03-11 19:19:19.00324266 +0000 UTC m=+0.402230534 container init 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-11T19:19:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:19 smithi143 ceph-mon[36013]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:19 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 podman[66802]: 2024-03-11 19:19:19.006010796 +0000 UTC m=+0.404998663 container start 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True) 2024-03-11T19:19:19.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 ceph-mds[66821]: -- 172.21.15.143:0/1137923795 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653faa3a1a0 con 0x5653faa10c00 2024-03-11T19:19:19.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: starting mds.c at 2024-03-11T19:19:19.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 bash[66802]: 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 2024-03-11T19:19:19.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:19:19 smithi143 systemd[1]: Started Ceph mds.c for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:19 smithi106 ceph-mon[35889]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:19 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.646 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:19.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:19 smithi072 ceph-mon[31695]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:19 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 -- 172.21.15.72:0/535169874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941089b0 msgr2=0x7f8f94108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 --2- 172.21.15.72:0/535169874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941089b0 0x7f8f94108e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009900 tx=0x7f8f8002bc50 comp rx=0 tx=0).stop 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 -- 172.21.15.72:0/535169874 shutdown_connections 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 --2- 172.21.15.72:0/535169874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f94109370 0x7f8f9410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 --2- 172.21.15.72:0/535169874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941089b0 0x7f8f94108e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:19.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 --2- 172.21.15.72:0/535169874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f941077c0 0x7f8f94107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 -- 172.21.15.72:0/535169874 >> 172.21.15.72:0/535169874 conn(0x7f8f94076790 msgr2=0x7f8f94078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 -- 172.21.15.72:0/535169874 shutdown_connections 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.870+0000 7f8f9c600640 1 -- 172.21.15.72:0/535169874 wait complete. 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.871+0000 7f8f9c600640 1 Processor -- start 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.871+0000 7f8f9c600640 1 -- start start 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.871+0000 7f8f9c600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 0x7f8f9419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.871+0000 7f8f9c600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f9c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f94109370 0x7f8f941a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f9c600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941a5320 con 0x7f8f941089b0 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f9c600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941a5490 con 0x7f8f941077c0 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f9c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f941a5650 con 0x7f8f94109370 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43254/0 (socket says 172.21.15.72:43254) 2024-03-11T19:19:19.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99200640 1 -- 172.21.15.72:0/1061653275 learned_addr learned my addr 172.21.15.72:0/1061653275 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 0x7f8f9419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f9a600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f94109370 0x7f8f941a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 -- 172.21.15.72:0/1061653275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f94109370 msgr2=0x7f8f941a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f94109370 0x7f8f941a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 -- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 msgr2=0x7f8f9419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99c00640 1 -- 172.21.15.72:0/1061653275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80009560 con 0x7f8f941077c0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.872+0000 7f8f99200640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f99c00640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 0x7f8f9419d830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9c0 tx=0x7f8f8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800ccc0 con 0x7f8f941077c0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f8800ce20 con 0x7f8f941077c0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f941a5930 con 0x7f8f941077c0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88010430 con 0x7f8f941077c0 2024-03-11T19:19:19.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.873+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f941a5e40 con 0x7f8f941077c0 2024-03-11T19:19:19.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.874+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9410ea80 con 0x7f8f941077c0 2024-03-11T19:19:19.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.876+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8f88004000 con 0x7f8f941077c0 2024-03-11T19:19:19.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.877+0000 7f8f92000640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 0x7f8f7c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:19.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.877+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8f88014070 con 0x7f8f941077c0 2024-03-11T19:19:19.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.877+0000 7f8f99200640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 0x7f8f7c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:19.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.878+0000 7f8f99200640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 0x7f8f7c0a3f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f9419ed40 tx=0x7f8f8002c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:19.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:19.881+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f880968f0 con 0x7f8f941077c0 2024-03-11T19:19:20.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.116+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f941131c0 con 0x7f8f941077c0 2024-03-11T19:19:20.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.117+0000 7f8f92000640 1 -- 172.21.15.72:0/1061653275 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f8f88099fb0 con 0x7f8f941077c0 2024-03-11T19:19:20.120 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:20.121 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-11T19:19:20.121 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 msgr2=0x7f8f7c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 0x7f8f7c0a3f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f9419ed40 tx=0x7f8f8002c800 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 msgr2=0x7f8f9419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 0x7f8f9419d830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9c0 tx=0x7f8f8800ee80 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 shutdown_connections 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8f7c0a1ab0 0x7f8f7c0a3f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f94109370 0x7f8f941a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f941089b0 0x7f8f9419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 --2- 172.21.15.72:0/1061653275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f941077c0 0x7f8f9419d830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 >> 172.21.15.72:0/1061653275 conn(0x7f8f94076790 msgr2=0x7f8f9410bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:20.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 shutdown_connections 2024-03-11T19:19:20.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:20.120+0000 7f8f9c600640 1 -- 172.21.15.72:0/1061653275 wait complete. 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6836/902566146,v1:172.21.15.143:6837/902566146] up:boot 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: fsmap 7 up:standby 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:20.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:20.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:20.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: Deploying daemon mds.h on smithi106 2024-03-11T19:19:20.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:20 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1061653275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:20.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6836/902566146,v1:172.21.15.143:6837/902566146] up:boot 2024-03-11T19:19:20.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: fsmap 7 up:standby 2024-03-11T19:19:20.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: Deploying daemon mds.h on smithi106 2024-03-11T19:19:20.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:20 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1061653275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6836/902566146,v1:172.21.15.143:6837/902566146] up:boot 2024-03-11T19:19:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: fsmap 7 up:standby 2024-03-11T19:19:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:19:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: Deploying daemon mds.h on smithi106 2024-03-11T19:19:20.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:20 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1061653275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:21.404 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:21 smithi106 ceph-mon[35889]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:21 smithi072 ceph-mon[31695]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:21 smithi143 ceph-mon[36013]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:22.145 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 -- 172.21.15.72:0/1796187663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c109190 msgr2=0x7f962c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 --2- 172.21.15.72:0/1796187663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c109190 0x7f962c10fa20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9624009900 tx=0x7f962402bc40 comp rx=0 tx=0).stop 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 -- 172.21.15.72:0/1796187663 shutdown_connections 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 --2- 172.21.15.72:0/1796187663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c109190 0x7f962c10fa20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 --2- 172.21.15.72:0/1796187663 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1087d0 0x7f962c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 --2- 172.21.15.72:0/1796187663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c1075e0 0x7f962c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.358+0000 7f962be00640 1 -- 172.21.15.72:0/1796187663 >> 172.21.15.72:0/1796187663 conn(0x7f962c076590 msgr2=0x7f962c0789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:22.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.359+0000 7f962be00640 1 -- 172.21.15.72:0/1796187663 shutdown_connections 2024-03-11T19:19:22.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.359+0000 7f962be00640 1 -- 172.21.15.72:0/1796187663 wait complete. 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 Processor -- start 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 -- start start 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c1087d0 0x7f962c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 0x7f962c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1a5090 con 0x7f962c1075e0 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1a5200 con 0x7f962c109190 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1a53c0 con 0x7f962c1087d0 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35946/0 (socket says 172.21.15.72:35946) 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962aa00640 1 -- 172.21.15.72:0/2722591668 learned_addr learned my addr 172.21.15.72:0/2722591668 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:22.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f962b400640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 0x7f962c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.360+0000 7f9622000640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c1087d0 0x7f962c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 -- 172.21.15.72:0/2722591668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c1087d0 msgr2=0x7f962c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c1087d0 0x7f962c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 -- 172.21.15.72:0/2722591668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 msgr2=0x7f962c1a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 0x7f962c1a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 -- 172.21.15.72:0/2722591668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c009840 con 0x7f962c1075e0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962b400640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 0x7f962c1a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962aa00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f961c009d20 tx=0x7f961c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c009b40 con 0x7f962c1075e0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f961c010b10 con 0x7f962c1075e0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c018cc0 con 0x7f962c1075e0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.361+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624009560 con 0x7f962c1075e0 2024-03-11T19:19:22.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.362+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c1a5a20 con 0x7f962c1075e0 2024-03-11T19:19:22.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.363+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c10e830 con 0x7f962c1075e0 2024-03-11T19:19:22.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.364+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f961c010c80 con 0x7f962c1075e0 2024-03-11T19:19:22.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.366+0000 7f9623e00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 0x7f96100a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:22.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.366+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f961c014070 con 0x7f962c1075e0 2024-03-11T19:19:22.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.366+0000 7f9622000640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 0x7f96100a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:22.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.367+0000 7f9622000640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 0x7f96100a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f962c19eab0 tx=0x7f9614005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:22.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.370+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f961c096f70 con 0x7f962c1075e0 2024-03-11T19:19:22.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.603+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f962c02ce40 con 0x7f962c1075e0 2024-03-11T19:19:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.605+0000 7f9623e00640 1 -- 172.21.15.72:0/2722591668 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f961c09a630 con 0x7f962c1075e0 2024-03-11T19:19:22.608 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:22.608 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-11T19:19:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 9 2024-03-11T19:19:22.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 msgr2=0x7f96100a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 0x7f96100a3ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f962c19eab0 tx=0x7f9614005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 msgr2=0x7f962c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f961c009d20 tx=0x7f961c00c6a0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 shutdown_connections 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f96100a1a30 0x7f96100a3ef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f962c109190 0x7f962c1a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f962c1087d0 0x7f962c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.607+0000 7f962be00640 1 --2- 172.21.15.72:0/2722591668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f962c1075e0 0x7f962c19d5a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.608+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 >> 172.21.15.72:0/2722591668 conn(0x7f962c076590 msgr2=0x7f962c10ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.608+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 shutdown_connections 2024-03-11T19:19:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:22.608+0000 7f962be00640 1 -- 172.21.15.72:0/2722591668 wait complete. 2024-03-11T19:19:23.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: Deploying daemon mds.i on smithi143 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6836/2982797341,v1:172.21.15.106:6837/2982797341] up:boot 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: fsmap 8 up:standby 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:19:23.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:23 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2722591668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: Deploying daemon mds.i on smithi143 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6836/2982797341,v1:172.21.15.106:6837/2982797341] up:boot 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: fsmap 8 up:standby 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:19:23.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:23 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2722591668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: Deploying daemon mds.i on smithi143 2024-03-11T19:19:23.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6836/2982797341,v1:172.21.15.106:6837/2982797341] up:boot 2024-03-11T19:19:23.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: fsmap 8 up:standby 2024-03-11T19:19:23.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:19:23.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:23 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2722591668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:24.439 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 systemd[1]: Starting Ceph mds.i for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:24.629 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:24.797 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 podman[67103]: 2024-03-11 19:19:24.383291365 +0000 UTC m=+0.017429037 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:24.797 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 podman[67103]: 2024-03-11 19:19:24.4914811 +0000 UTC m=+0.125618764 container create b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397) 2024-03-11T19:19:24.797 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 podman[67103]: 2024-03-11 19:19:24.794201304 +0000 UTC m=+0.428338957 container init b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-11T19:19:24.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 -- 172.21.15.72:0/338706015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc1033c0 msgr2=0x7fa8dc1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:24.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/338706015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc1037e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009900 tx=0x7fa8cc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:24.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 -- 172.21.15.72:0/338706015 shutdown_connections 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/338706015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc104f70 0x7fa8dc10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/338706015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1045b0 0x7fa8dc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/338706015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc1037e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 -- 172.21.15.72:0/338706015 >> 172.21.15.72:0/338706015 conn(0x7fa8dc0fe970 msgr2=0x7fa8dc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 -- 172.21.15.72:0/338706015 shutdown_connections 2024-03-11T19:19:24.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.845+0000 7fa8e3a00640 1 -- 172.21.15.72:0/338706015 wait complete. 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.846+0000 7fa8e3a00640 1 Processor -- start 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.846+0000 7fa8e3a00640 1 -- start start 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.846+0000 7fa8e3a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.846+0000 7fa8e3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc1045b0 0x7fa8dc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.846+0000 7fa8e3a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:24.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e3a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a0f50 con 0x7fa8dc104f70 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e3a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a10c0 con 0x7fa8dc1033c0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a1280 con 0x7fa8dc1045b0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8d3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc1045b0 0x7fa8dc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35970/0 (socket says 172.21.15.72:35970) 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 -- 172.21.15.72:0/1654854272 learned_addr learned my addr 172.21.15.72:0/1654854272 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1000640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 -- 172.21.15.72:0/1654854272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc1045b0 msgr2=0x7fa8dc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc1045b0 0x7fa8dc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 -- 172.21.15.72:0/1654854272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 msgr2=0x7fa8dc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1a00640 1 -- 172.21.15.72:0/1654854272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009490 con 0x7fa8dc104f70 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.847+0000 7fa8e1000640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8e1a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400eb10 tx=0x7fa8d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:24.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4004b20 con 0x7fa8dc104f70 2024-03-11T19:19:24.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8d4004c80 con 0x7fa8dc104f70 2024-03-11T19:19:24.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc009560 con 0x7fa8dc104f70 2024-03-11T19:19:24.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4010650 con 0x7fa8dc104f70 2024-03-11T19:19:24.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.848+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc1a1890 con 0x7fa8dc104f70 2024-03-11T19:19:24.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.849+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc10a6e0 con 0x7fa8dc104f70 2024-03-11T19:19:24.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.851+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa8d4005020 con 0x7fa8dc104f70 2024-03-11T19:19:24.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.852+0000 7fa8da000640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 0x7fa8c00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:24.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.852+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8d4014070 con 0x7fa8dc104f70 2024-03-11T19:19:24.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.852+0000 7fa8e1000640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 0x7fa8c00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:24.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.853+0000 7fa8e1000640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 0x7fa8c00a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc02c240 tx=0x7fa8cc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:24.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:24.856+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8d4096bd0 con 0x7fa8dc104f70 2024-03-11T19:19:25.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.095+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8dc02ce40 con 0x7fa8dc104f70 2024-03-11T19:19:25.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.096+0000 7fa8da000640 1 -- 172.21.15.72:0/1654854272 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7fa8d409a290 con 0x7fa8dc104f70 2024-03-11T19:19:25.099 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:25.099 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-11T19:19:25.099 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 9 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 msgr2=0x7fa8c00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 0x7fa8c00a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc02c240 tx=0x7fa8cc02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 msgr2=0x7fa8dc1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400eb10 tx=0x7fa8d400c2d0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 shutdown_connections 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa8c00a1a30 0x7fa8c00a3ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8dc104f70 0x7fa8dc1a0a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc1045b0 0x7fa8dc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 --2- 172.21.15.72:0/1654854272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8dc1033c0 0x7fa8dc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:25.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 >> 172.21.15.72:0/1654854272 conn(0x7fa8dc0fe970 msgr2=0x7fa8dc1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:25.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 shutdown_connections 2024-03-11T19:19:25.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:25.099+0000 7fa8e3a00640 1 -- 172.21.15.72:0/1654854272 wait complete. 2024-03-11T19:19:25.103 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 podman[67103]: 2024-03-11 19:19:24.797196444 +0000 UTC m=+0.431334115 container start b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:19:25.103 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 ceph-mds[67122]: -- 172.21.15.143:0/2588091351 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619649381a0 con 0x56196490f000 2024-03-11T19:19:25.103 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: starting mds.i at 2024-03-11T19:19:25.103 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 bash[67103]: b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b 2024-03-11T19:19:25.103 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:19:24 smithi143 systemd[1]: Started Ceph mds.i for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:25.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:25.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:25.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:25 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1654854272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:25.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:25.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:25.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:25 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1654854272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:25.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:25.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:25 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1654854272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:26.318 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:26 smithi072 ceph-mon[31695]: Deploying daemon mds.g on smithi072 2024-03-11T19:19:26.319 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:26 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:boot 2024-03-11T19:19:26.319 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:26 smithi072 ceph-mon[31695]: fsmap 9 up:standby 2024-03-11T19:19:26.319 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:26 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:19:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:26 smithi106 ceph-mon[35889]: Deploying daemon mds.g on smithi072 2024-03-11T19:19:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:26 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:boot 2024-03-11T19:19:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:26 smithi106 ceph-mon[35889]: fsmap 9 up:standby 2024-03-11T19:19:26.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:26 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:19:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:26 smithi143 ceph-mon[36013]: Deploying daemon mds.g on smithi072 2024-03-11T19:19:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:26 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:boot 2024-03-11T19:19:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:26 smithi143 ceph-mon[36013]: fsmap 9 up:standby 2024-03-11T19:19:26.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:26 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:19:27.124 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:27.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:27 smithi072 ceph-mon[31695]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:27.251 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:27 smithi143 ceph-mon[36013]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 -- 172.21.15.72:0/994063749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c1000a0 msgr2=0x7f916c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 --2- 172.21.15.72:0/994063749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c1000a0 0x7f916c1004c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c02bc50 comp rx=0 tx=0).stop 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 -- 172.21.15.72:0/994063749 shutdown_connections 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 --2- 172.21.15.72:0/994063749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 --2- 172.21.15.72:0/994063749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c101290 0x7f916c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 --2- 172.21.15.72:0/994063749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c1000a0 0x7f916c1004c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 -- 172.21.15.72:0/994063749 >> 172.21.15.72:0/994063749 conn(0x7f916c0fb610 msgr2=0x7f916c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 -- 172.21.15.72:0/994063749 shutdown_connections 2024-03-11T19:19:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.347+0000 7f9171600640 1 -- 172.21.15.72:0/994063749 wait complete. 2024-03-11T19:19:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 Processor -- start 2024-03-11T19:19:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 -- start start 2024-03-11T19:19:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 0x7f916c102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c103310 con 0x7f916c1000a0 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c103480 con 0x7f916c101c50 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.348+0000 7f9171600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1a12e0 con 0x7f916c101290 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35984/0 (socket says 172.21.15.72:35984) 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 -- 172.21.15.72:0/3891059355 learned_addr learned my addr 172.21.15.72:0/3891059355 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916a000640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 0x7f916c102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916b400640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 -- 172.21.15.72:0/3891059355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 msgr2=0x7f916c102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 0x7f916c102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 -- 172.21.15.72:0/3891059355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 msgr2=0x7f916c102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916aa00640 1 -- 172.21.15.72:0/3891059355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009490 con 0x7f916c1000a0 2024-03-11T19:19:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916b400640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.349+0000 7f916a000640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 0x7f916c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f916aa00640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f915c000c00 tx=0x7f915c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c0043d0 con 0x7f916c1000a0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f915c0052d0 con 0x7f916c1000a0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c009560 con 0x7f916c1000a0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c003d70 con 0x7f916c1000a0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.350+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c1a18b0 con 0x7f916c1000a0 2024-03-11T19:19:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.351+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c10bab0 con 0x7f916c1000a0 2024-03-11T19:19:27.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.353+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f915c005440 con 0x7f916c1000a0 2024-03-11T19:19:27.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.355+0000 7f9163400640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 0x7f91500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:27.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.355+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f915c0ef100 con 0x7f916c1000a0 2024-03-11T19:19:27.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.355+0000 7f916a000640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 0x7f91500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:27.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.356+0000 7f916a000640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 0x7f91500a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f916c103da0 tx=0x7f9158005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:27.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.360+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f915c0b8710 con 0x7f916c1000a0 2024-03-11T19:19:27.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:27 smithi106 ceph-mon[35889]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.600+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f916c10bc90 con 0x7f916c1000a0 2024-03-11T19:19:27.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.602+0000 7f9163400640 1 -- 172.21.15.72:0/3891059355 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f915c0bbdd0 con 0x7f916c1000a0 2024-03-11T19:19:27.605 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:27.605 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:19:27.605 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 10 2024-03-11T19:19:27.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 msgr2=0x7f91500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 0x7f91500a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f916c103da0 tx=0x7f9158005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 msgr2=0x7f916c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f915c000c00 tx=0x7f915c005e50 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 shutdown_connections 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f91500a1a30 0x7f91500a3ef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f916c101c50 0x7f916c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f916c101290 0x7f916c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 --2- 172.21.15.72:0/3891059355 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f916c1000a0 0x7f916c104030 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.604+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 >> 172.21.15.72:0/3891059355 conn(0x7f916c0fb610 msgr2=0x7f916c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.605+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 shutdown_connections 2024-03-11T19:19:27.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:27.605+0000 7f9171600640 1 -- 172.21.15.72:0/3891059355 wait complete. 2024-03-11T19:19:28.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:28 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3891059355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:28.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:28 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3891059355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:28.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:28 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3891059355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:28.634 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:28 smithi072 systemd[1]: Starting Ceph mds.g for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:28.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:28 smithi072 podman[73113]: 2024-03-11 19:19:28.53638827 +0000 UTC m=+0.043828285 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:28.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:28 smithi072 podman[73113]: 2024-03-11 19:19:28.682652258 +0000 UTC m=+0.190092308 container create a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True) 2024-03-11T19:19:29.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:29 smithi106 ceph-mon[35889]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:29.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:29 smithi072 ceph-mon[31695]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:28 smithi072 podman[73113]: 2024-03-11 19:19:28.99903765 +0000 UTC m=+0.506477704 container init a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:29 smithi072 podman[73113]: 2024-03-11 19:19:29.008283848 +0000 UTC m=+0.515723869 container start a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397) 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:29 smithi072 ceph-mds[73140]: -- 172.21.15.72:0/1806529638 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cde46421a0 con 0x55cde4618800 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: starting mds.g at 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:29 smithi072 bash[73113]: a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 2024-03-11T19:19:29.430 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:19:29 smithi072 systemd[1]: Started Ceph mds.g for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:29.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:29 smithi143 ceph-mon[36013]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:29.631 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:29.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.831+0000 7f43eaa00640 1 -- 172.21.15.72:0/2295789035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e40751c0 msgr2=0x7f43e4073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:29.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.831+0000 7f43eaa00640 1 --2- 172.21.15.72:0/2295789035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e40751c0 0x7f43e4073610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009860 tx=0x7f43d802bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 -- 172.21.15.72:0/2295789035 shutdown_connections 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 --2- 172.21.15.72:0/2295789035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e4108ea0 0x7f43e410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 --2- 172.21.15.72:0/2295789035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4073be0 0x7f43e4074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 --2- 172.21.15.72:0/2295789035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e40751c0 0x7f43e4073610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 -- 172.21.15.72:0/2295789035 >> 172.21.15.72:0/2295789035 conn(0x7f43e40fc550 msgr2=0x7f43e40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 -- 172.21.15.72:0/2295789035 shutdown_connections 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.832+0000 7f43eaa00640 1 -- 172.21.15.72:0/2295789035 wait complete. 2024-03-11T19:19:29.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 Processor -- start 2024-03-11T19:19:29.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 -- start start 2024-03-11T19:19:29.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:29.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e40751c0 0x7f43e41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:29.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 0x7f43e41a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a0d60 con 0x7f43e4073be0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a0ed0 con 0x7f43e4108ea0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43eaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a1090 con 0x7f43e40751c0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43e9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43ea000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 0x7f43e41a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43e9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36026/0 (socket says 172.21.15.72:36026) 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43e9600640 1 -- 172.21.15.72:0/4234384146 learned_addr learned my addr 172.21.15.72:0/4234384146 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.833+0000 7f43e8c00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e40751c0 0x7f43e41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 -- 172.21.15.72:0/4234384146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e40751c0 msgr2=0x7f43e41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e40751c0 0x7f43e41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:29.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 -- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 msgr2=0x7f43e4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 -- 172.21.15.72:0/4234384146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8009490 con 0x7f43e4108ea0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43e9600640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43ea000640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 0x7f43e41a0820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ef00 tx=0x7f43dc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc004b20 con 0x7f43e4108ea0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43dc003e50 con 0x7f43e4108ea0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00b570 con 0x7f43e4108ea0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.834+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e41a1370 con 0x7f43e4108ea0 2024-03-11T19:19:29.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.835+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e41a1830 con 0x7f43e4108ea0 2024-03-11T19:19:29.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.836+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4107bd0 con 0x7f43e4108ea0 2024-03-11T19:19:29.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.839+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f43dc00a040 con 0x7f43e4108ea0 2024-03-11T19:19:29.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.840+0000 7f43e2000640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 0x7f43cc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:29.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.841+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43dc0ccff0 con 0x7f43e4108ea0 2024-03-11T19:19:29.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.841+0000 7f43e9600640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 0x7f43cc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:29.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.841+0000 7f43e9600640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 0x7f43cc0a3f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c00 tx=0x7f43d8034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:29.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:29.842+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43dc096640 con 0x7f43e4108ea0 2024-03-11T19:19:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.090+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43e410ebe0 con 0x7f43e4108ea0 2024-03-11T19:19:30.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.091+0000 7f43e2000640 1 -- 172.21.15.72:0/4234384146 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f43dc099d00 con 0x7f43e4108ea0 2024-03-11T19:19:30.095 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:30.095 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-11T19:19:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 11 2024-03-11T19:19:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 msgr2=0x7f43cc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 0x7f43cc0a3f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c00 tx=0x7f43d8034be0 comp rx=0 tx=0).stop 2024-03-11T19:19:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 msgr2=0x7f43e41a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 0x7f43e41a0820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ef00 tx=0x7f43dc00c740 comp rx=0 tx=0).stop 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 shutdown_connections 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43cc0a1ab0 0x7f43cc0a3f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e4108ea0 0x7f43e41a0820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43e40751c0 0x7f43e41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 --2- 172.21.15.72:0/4234384146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4073be0 0x7f43e4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 >> 172.21.15.72:0/4234384146 conn(0x7f43e40fc550 msgr2=0x7f43e4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 shutdown_connections 2024-03-11T19:19:30.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:30.094+0000 7f43eaa00640 1 -- 172.21.15.72:0/4234384146 wait complete. 2024-03-11T19:19:30.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:boot 2024-03-11T19:19:30.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: fsmap 10 up:standby 2024-03-11T19:19:30.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:19:30.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:30.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:30.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:30.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:30 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/4234384146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:boot 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: fsmap 10 up:standby 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:30 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4234384146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:boot 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: fsmap 10 up:standby 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:30.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:30.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:30.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:30 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/4234384146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:31 smithi106 ceph-mon[35889]: Deploying daemon mds.a on smithi072 2024-03-11T19:19:31.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:31 smithi106 ceph-mon[35889]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:31 smithi072 ceph-mon[31695]: Deploying daemon mds.a on smithi072 2024-03-11T19:19:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:31 smithi072 ceph-mon[31695]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:31.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:31 smithi143 ceph-mon[36013]: Deploying daemon mds.a on smithi072 2024-03-11T19:19:31.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:31 smithi143 ceph-mon[36013]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:32.115 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 -- 172.21.15.72:0/2660058269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 msgr2=0x7f638010d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 --2- 172.21.15.72:0/2660058269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f638010d1c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6370009860 tx=0x7f637002bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 -- 172.21.15.72:0/2660058269 shutdown_connections 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 --2- 172.21.15.72:0/2660058269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f638010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 --2- 172.21.15.72:0/2660058269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f638010d1c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 --2- 172.21.15.72:0/2660058269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f638010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 -- 172.21.15.72:0/2660058269 >> 172.21.15.72:0/2660058269 conn(0x7f638006dff0 msgr2=0x7f6380070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 -- 172.21.15.72:0/2660058269 shutdown_connections 2024-03-11T19:19:32.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.389+0000 7f6386600640 1 -- 172.21.15.72:0/2660058269 wait complete. 2024-03-11T19:19:32.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.390+0000 7f6386600640 1 Processor -- start 2024-03-11T19:19:32.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.390+0000 7f6386600640 1 -- start start 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f63801a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f63801a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f63801a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a9880 con 0x7f638010d850 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a99f0 con 0x7f63801083f0 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f6386600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a9bb0 con 0x7f638010adb0 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f637fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f63801a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f637fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f63801a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60952/0 (socket says 172.21.15.72:60952) 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f637f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f63801a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f637fe00640 1 -- 172.21.15.72:0/3971243042 learned_addr learned my addr 172.21.15.72:0/3971243042 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.391+0000 7f637ea00640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f63801a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 -- 172.21.15.72:0/3971243042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 msgr2=0x7f63801a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f63801a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 -- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 msgr2=0x7f63801a9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f63801a9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 -- 172.21.15.72:0/3971243042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370009490 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f637f400640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f63801a1df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f636c009fd0 tx=0x7f636c00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00ccc0 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f636c00ee70 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63801a9e90 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c004a00 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.392+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801aa260 con 0x7f63801083f0 2024-03-11T19:19:32.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.393+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638006fae0 con 0x7f63801083f0 2024-03-11T19:19:32.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.396+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f636c004b60 con 0x7f63801083f0 2024-03-11T19:19:32.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.397+0000 7f6377e00640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 0x7f63640a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:32.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.397+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f636c014070 con 0x7f63801083f0 2024-03-11T19:19:32.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.397+0000 7f637ea00640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 0x7f63640a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:32.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.398+0000 7f637ea00640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 0x7f63640a3ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63700057c0 tx=0x7f6370034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:32.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.401+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f636c0964e0 con 0x7f63801083f0 2024-03-11T19:19:32.610 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:32 smithi072 systemd[1]: Starting Ceph mds.a for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:32.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.644+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6380114540 con 0x7f63801083f0 2024-03-11T19:19:32.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.645+0000 7f6377e00640 1 -- 172.21.15.72:0/3971243042 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f636c098410 con 0x7f63801083f0 2024-03-11T19:19:32.648 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:32.648 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-11T19:19:32.648 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 11 2024-03-11T19:19:32.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 msgr2=0x7f63640a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 0x7f63640a3ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63700057c0 tx=0x7f6370034be0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 msgr2=0x7f63801a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f63801a1df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f636c009fd0 tx=0x7f636c00ec20 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 shutdown_connections 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63640a1a10 0x7f63640a3ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f638010d850 0x7f63801a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f638010adb0 0x7f63801a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 --2- 172.21.15.72:0/3971243042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63801083f0 0x7f63801a1df0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.648+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 >> 172.21.15.72:0/3971243042 conn(0x7f638006dff0 msgr2=0x7f63801094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.649+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 shutdown_connections 2024-03-11T19:19:32.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:32.649+0000 7f6386600640 1 -- 172.21.15.72:0/3971243042 wait complete. 2024-03-11T19:19:33.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:33 smithi072 ceph-mon[31695]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:33.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:33 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3971243042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:33.336 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:32 smithi072 podman[73505]: 2024-03-11 19:19:32.879839645 +0000 UTC m=+0.035163206 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:33.337 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 podman[73505]: 2024-03-11 19:19:33.026007612 +0000 UTC m=+0.181331209 container create 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-11T19:19:33.337 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 podman[73505]: 2024-03-11 19:19:33.331824717 +0000 UTC m=+0.487148321 container init 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-11T19:19:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:33 smithi143 ceph-mon[36013]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:33 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3971243042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:33 smithi106 ceph-mon[35889]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3971243042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:33.680 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 podman[73505]: 2024-03-11 19:19:33.343490852 +0000 UTC m=+0.498814483 container start 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-11T19:19:33.680 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 ceph-mds[73524]: -- 172.21.15.72:0/4023675405 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f320dc1a0 con 0x563f320b2800 2024-03-11T19:19:33.680 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a[73520]: starting mds.a at 2024-03-11T19:19:33.680 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 bash[73505]: 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d 2024-03-11T19:19:33.681 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:19:33 smithi072 systemd[1]: Started Ceph mds.a for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:34.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:34.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 -- 172.21.15.72:0/1359656786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4100fe0 msgr2=0x7fdfd4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/1359656786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4101400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009860 tx=0x7fdfc402bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 -- 172.21.15.72:0/1359656786 shutdown_connections 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/1359656786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4102b90 0x7fdfd4109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/1359656786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd41021d0 0x7fdfd4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/1359656786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4101400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 -- 172.21.15.72:0/1359656786 >> 172.21.15.72:0/1359656786 conn(0x7fdfd40fc550 msgr2=0x7fdfd40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 -- 172.21.15.72:0/1359656786 shutdown_connections 2024-03-11T19:19:34.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.892+0000 7fdfd8a00640 1 -- 172.21.15.72:0/1359656786 wait complete. 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:34.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: Deploying daemon mds.b on smithi106 2024-03-11T19:19:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6840/1103846445,v1:172.21.15.72:6841/1103846445] up:boot 2024-03-11T19:19:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: fsmap 11 up:standby 2024-03-11T19:19:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:34 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:19:34.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.893+0000 7fdfd8a00640 1 Processor -- start 2024-03-11T19:19:34.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.893+0000 7fdfd8a00640 1 -- start start 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 0x7fdfd41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41a0df0 con 0x7fdfd4100fe0 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41a0f60 con 0x7fdfd41021d0 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41a1120 con 0x7fdfd4102b90 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52136/0 (socket says 172.21.15.72:52136) 2024-03-11T19:19:34.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60994/0 (socket says 172.21.15.72:60994) 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 -- 172.21.15.72:0/2896918300 learned_addr learned my addr 172.21.15.72:0/2896918300 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd3200640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 0x7fdfd41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 -- 172.21.15.72:0/2896918300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 msgr2=0x7fdfd41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 0x7fdfd41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 -- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 msgr2=0x7fdfd4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfcbe00640 1 -- 172.21.15.72:0/2896918300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009900 con 0x7fdfd41021d0 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.894+0000 7fdfd3200640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 0x7fdfd41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfcbe00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9f0 tx=0x7fdfbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:34.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc009700 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc00cd80 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc4009490 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc018920 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.895+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd41a1780 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.896+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd410c9b0 con 0x7fdfd41021d0 2024-03-11T19:19:34.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.898+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdfbc018a80 con 0x7fdfd41021d0 2024-03-11T19:19:34.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.899+0000 7fdfcb400640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 0x7fdfb80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:34.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.899+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfbc014070 con 0x7fdfd41021d0 2024-03-11T19:19:34.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.899+0000 7fdfd2800640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 0x7fdfb80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:34.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.900+0000 7fdfd2800640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 0x7fdfb80a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009830 tx=0x7fdfc4034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:34.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:34.903+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdfbc052d20 con 0x7fdfd41021d0 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: Deploying daemon mds.b on smithi106 2024-03-11T19:19:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6840/1103846445,v1:172.21.15.72:6841/1103846445] up:boot 2024-03-11T19:19:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: fsmap 11 up:standby 2024-03-11T19:19:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:34 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:19:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: Deploying daemon mds.b on smithi106 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6840/1103846445,v1:172.21.15.72:6841/1103846445] up:boot 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: fsmap 11 up:standby 2024-03-11T19:19:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:34 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:19:35.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.137+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfd410cb40 con 0x7fdfd41021d0 2024-03-11T19:19:35.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.139+0000 7fdfcb400640 1 -- 172.21.15.72:0/2896918300 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7fdfbc096a30 con 0x7fdfd41021d0 2024-03-11T19:19:35.142 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:35.142 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-11T19:19:35.142 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 12 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 msgr2=0x7fdfb80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 0x7fdfb80a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009830 tx=0x7fdfc4034c00 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 msgr2=0x7fdfd4199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9f0 tx=0x7fdfbc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 shutdown_connections 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdfb80a1a30 0x7fdfb80a3ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd4102b90 0x7fdfd41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd41021d0 0x7fdfd4199890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 --2- 172.21.15.72:0/2896918300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfd4100fe0 0x7fdfd4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.141+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 >> 172.21.15.72:0/2896918300 conn(0x7fdfd40fc550 msgr2=0x7fdfd40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.142+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 shutdown_connections 2024-03-11T19:19:35.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:35.142+0000 7fdfd8a00640 1 -- 172.21.15.72:0/2896918300 wait complete. 2024-03-11T19:19:35.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:35 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2896918300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:35.896 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:35 smithi106 systemd[1]: Starting Ceph mds.b for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:19:35.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:35 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2896918300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:35.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:35 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2896918300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:36.237 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:35 smithi106 podman[66276]: 2024-03-11 19:19:35.867642165 +0000 UTC m=+0.017483263 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:19:36.237 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:35 smithi106 podman[66276]: 2024-03-11 19:19:35.981055852 +0000 UTC m=+0.130896986 container create 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:19:36.237 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 podman[66276]: 2024-03-11 19:19:36.23394947 +0000 UTC m=+0.383790555 container init 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-11T19:19:36.645 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 podman[66276]: 2024-03-11 19:19:36.236651045 +0000 UTC m=+0.386492139 container start 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:19:36.645 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mds[66295]: -- 172.21.15.106:0/450384668 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624dc1781a0 con 0x5624dc14ec00 2024-03-11T19:19:36.645 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: starting mds.b at 2024-03-11T19:19:36.645 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 bash[66276]: 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f 2024-03-11T19:19:36.646 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:36 smithi106 systemd[1]: Started Ceph mds.b for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:36 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:36 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.163 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:37.179 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:37.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:36 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:37.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 -- 172.21.15.72:0/3068753330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c1033c0 msgr2=0x7f0a0c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 --2- 172.21.15.72:0/3068753330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1037e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009860 tx=0x7f09fc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 -- 172.21.15.72:0/3068753330 shutdown_connections 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 --2- 172.21.15.72:0/3068753330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c104f70 0x7f0a0c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 --2- 172.21.15.72:0/3068753330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 0x7f0a0c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 --2- 172.21.15.72:0/3068753330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1037e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 -- 172.21.15.72:0/3068753330 >> 172.21.15.72:0/3068753330 conn(0x7f0a0c0fe970 msgr2=0x7f0a0c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 -- 172.21.15.72:0/3068753330 shutdown_connections 2024-03-11T19:19:37.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.368+0000 7f0a12400640 1 -- 172.21.15.72:0/3068753330 wait complete. 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.369+0000 7f0a12400640 1 Processor -- start 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.369+0000 7f0a12400640 1 -- start start 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 0x7f0a0c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c104f70 0x7f0a0c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a0f80 con 0x7f0a0c1033c0 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a10f0 con 0x7f0a0c104f70 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a12400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a12b0 con 0x7f0a0c1045b0 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:32772/0 (socket says 172.21.15.72:32772) 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 -- 172.21.15.72:0/778449671 learned_addr learned my addr 172.21.15.72:0/778449671 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:37.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a03e00640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 0x7f0a0c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0be00640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c104f70 0x7f0a0c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 -- 172.21.15.72:0/778449671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 msgr2=0x7f0a0c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 0x7f0a0c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 -- 172.21.15.72:0/778449671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c104f70 msgr2=0x7f0a0c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c104f70 0x7f0a0c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.370+0000 7f0a0b400640 1 -- 172.21.15.72:0/778449671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc009490 con 0x7f0a0c1033c0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a0b400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02c0b0 tx=0x7f09fc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc02c510 con 0x7f0a0c1033c0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09fc02cb50 con 0x7f0a0c1033c0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c1a1530 con 0x7f0a0c1033c0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc003850 con 0x7f0a0c1033c0 2024-03-11T19:19:37.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.371+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1a18a0 con 0x7f0a0c1033c0 2024-03-11T19:19:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.373+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c10a6e0 con 0x7f0a0c1033c0 2024-03-11T19:19:37.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.374+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f09fc03b040 con 0x7f0a0c1033c0 2024-03-11T19:19:37.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.376+0000 7f0a08c00640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 0x7f09f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.376+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09fc0eeae0 con 0x7f0a0c1033c0 2024-03-11T19:19:37.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.376+0000 7f0a03e00640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 0x7f09f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.377+0000 7f0a03e00640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 0x7f09f00a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c19a9a0 tx=0x7f09f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:37.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.380+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09fc0b8220 con 0x7f0a0c1033c0 2024-03-11T19:19:37.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.624+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a0c02cdd0 con 0x7f0a0c1033c0 2024-03-11T19:19:37.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.624+0000 7f0a08c00640 1 -- 172.21.15.72:0/778449671 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7f09fc0bb8e0 con 0x7f0a0c1033c0 2024-03-11T19:19:37.629 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:37.629 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24572,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-11T19:19:37.629 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 13 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 msgr2=0x7f09f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 0x7f09f00a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c19a9a0 tx=0x7f09f4005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 msgr2=0x7f0a0c1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02c0b0 tx=0x7f09fc005d50 comp rx=0 tx=0).stop 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 shutdown_connections 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09f00a1a30 0x7f09f00a3ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a0c104f70 0x7f0a0c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a0c1045b0 0x7f0a0c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 --2- 172.21.15.72:0/778449671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a0c1033c0 0x7f0a0c1994e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.627+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 >> 172.21.15.72:0/778449671 conn(0x7f0a0c0fe970 msgr2=0x7f0a0c1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:37.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.628+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 shutdown_connections 2024-03-11T19:19:37.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.628+0000 7f0a12400640 1 -- 172.21.15.72:0/778449671 wait complete. 2024-03-11T19:19:37.652 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:19:37.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:37 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:boot 2024-03-11T19:19:37.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:37 smithi106 ceph-mon[35889]: fsmap 12 up:standby 2024-03-11T19:19:37.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:19:37.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:37 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:37.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:37 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/778449671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:37.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 -- 172.21.15.72:0/1832580244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 msgr2=0x7fe89810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 --2- 172.21.15.72:0/1832580244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe89810b800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe890009be0 tx=0x7fe89002bef0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 -- 172.21.15.72:0/1832580244 shutdown_connections 2024-03-11T19:19:37.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 --2- 172.21.15.72:0/1832580244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe89810b800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 --2- 172.21.15.72:0/1832580244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 --2- 172.21.15.72:0/1832580244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 0x7fe8981037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 -- 172.21.15.72:0/1832580244 >> 172.21.15.72:0/1832580244 conn(0x7fe8980fe930 msgr2=0x7fe898100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:37.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 -- 172.21.15.72:0/1832580244 shutdown_connections 2024-03-11T19:19:37.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.874+0000 7fe89d400640 1 -- 172.21.15.72:0/1832580244 wait complete. 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 Processor -- start 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 -- start start 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 0x7fe898199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981a0f10 con 0x7fe8981045b0 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981a1080 con 0x7fe898104f70 2024-03-11T19:19:37.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe89d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981a1240 con 0x7fe8981033c0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe88de00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe897200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe897200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52176/0 (socket says 172.21.15.72:52176) 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe897200640 1 -- 172.21.15.72:0/1579249972 learned_addr learned my addr 172.21.15.72:0/1579249972 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.875+0000 7fe896800640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 0x7fe898199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 -- 172.21.15.72:0/1579249972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 msgr2=0x7fe898199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 0x7fe898199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 -- 172.21.15.72:0/1579249972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 msgr2=0x7fe8981a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 -- 172.21.15.72:0/1579249972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890009840 con 0x7fe8981045b0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe897200640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88de00640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898199960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e790 tx=0x7fe88000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:37.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000ccc0 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.876+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981a1520 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.877+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe88000eea0 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.877+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880018730 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.877+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8981a1a80 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.878+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89810a680 con 0x7fe8981045b0 2024-03-11T19:19:37.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.880+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe880010430 con 0x7fe8981045b0 2024-03-11T19:19:37.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.882+0000 7fe88fe00640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 0x7fe87c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:37.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.882+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe880014070 con 0x7fe8981045b0 2024-03-11T19:19:37.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.882+0000 7fe896800640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 0x7fe87c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:37.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.883+0000 7fe896800640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 0x7fe87c0a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe898104410 tx=0x7fe888005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:37.885+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe880052d20 con 0x7fe8981045b0 2024-03-11T19:19:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:37 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:boot 2024-03-11T19:19:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:37 smithi072 ceph-mon[31695]: fsmap 12 up:standby 2024-03-11T19:19:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:19:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:37 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:37 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/778449671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:37 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:boot 2024-03-11T19:19:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:37 smithi143 ceph-mon[36013]: fsmap 12 up:standby 2024-03-11T19:19:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:19:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:37 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:37 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/778449671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:38.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.087+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe89810a0f0 con 0x7fe8981045b0 2024-03-11T19:19:38.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.089+0000 7fe88fe00640 1 -- 172.21.15.72:0/1579249972 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fe880096870 con 0x7fe8981045b0 2024-03-11T19:19:38.093 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:38.094 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":82,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:18:45.053935+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:19:38.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 msgr2=0x7fe87c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 0x7fe87c0a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe898104410 tx=0x7fe888005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:38.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 msgr2=0x7fe898199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898199960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e790 tx=0x7fe88000ec50 comp rx=0 tx=0).stop 2024-03-11T19:19:38.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 shutdown_connections 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe87c0a1a30 0x7fe87c0a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe898104f70 0x7fe8981a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8981045b0 0x7fe898199960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 --2- 172.21.15.72:0/1579249972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8981033c0 0x7fe898199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 >> 172.21.15.72:0/1579249972 conn(0x7fe8980fe930 msgr2=0x7fe898107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 shutdown_connections 2024-03-11T19:19:38.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.092+0000 7fe89d400640 1 -- 172.21.15.72:0/1579249972 wait complete. 2024-03-11T19:19:38.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:19:38.116 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.320+0000 7f640ea00640 1 -- 172.21.15.72:0/1475031238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 msgr2=0x7f6408100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.320+0000 7f640ea00640 1 --2- 172.21.15.72:0/1475031238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408100690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009900 tx=0x7f63f402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 -- 172.21.15.72:0/1475031238 shutdown_connections 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 --2- 172.21.15.72:0/1475031238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f6408101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 --2- 172.21.15.72:0/1475031238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408100690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 --2- 172.21.15.72:0/1475031238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64080ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 -- 172.21.15.72:0/1475031238 >> 172.21.15.72:0/1475031238 conn(0x7f64080fb4f0 msgr2=0x7f64080fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 -- 172.21.15.72:0/1475031238 shutdown_connections 2024-03-11T19:19:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.321+0000 7f640ea00640 1 -- 172.21.15.72:0/1475031238 wait complete. 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 Processor -- start 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 -- start start 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f64081a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a0db0 con 0x7f6408100bd0 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a0f20 con 0x7f6408100230 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.322+0000 7f640ea00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a10e0 con 0x7f64080ff8d0 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:53390/0 (socket says 172.21.15.72:53390) 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 -- 172.21.15.72:0/1223878365 learned_addr learned my addr 172.21.15.72:0/1223878365 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640cc00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640e000640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f64081a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 -- 172.21.15.72:0/1223878365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 msgr2=0x7f6408199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 -- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 msgr2=0x7f64081a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f64081a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 -- 172.21.15.72:0/1223878365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4009560 con 0x7f64080ff8d0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640e000640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f64081a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640cc00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.323+0000 7f640d600640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9f0 tx=0x7f63f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.324+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8009700 con 0x7f64080ff8d0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.324+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63f800cd80 con 0x7f64080ff8d0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.324+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64081a13c0 con 0x7f64080ff8d0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.324+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8018920 con 0x7f64080ff8d0 2024-03-11T19:19:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.324+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64081a18a0 con 0x7f64080ff8d0 2024-03-11T19:19:38.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.325+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640810a620 con 0x7f64080ff8d0 2024-03-11T19:19:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.327+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f63f8018a80 con 0x7f64080ff8d0 2024-03-11T19:19:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.328+0000 7f6406000640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 0x7f63ec0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.328+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63f8014070 con 0x7f64080ff8d0 2024-03-11T19:19:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.329+0000 7f640cc00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 0x7f63ec0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.329+0000 7f640cc00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 0x7f63ec0a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:38.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.334+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63f8052d20 con 0x7f64080ff8d0 2024-03-11T19:19:38.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.559+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f640802cdd0 con 0x7f64080ff8d0 2024-03-11T19:19:38.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.560+0000 7f6406000640 1 -- 172.21.15.72:0/1223878365 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63f8096a30 con 0x7f64080ff8d0 2024-03-11T19:19:38.564 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.562+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 msgr2=0x7f63ec0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.562+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 0x7f63ec0a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f402c7e0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.562+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 msgr2=0x7f64081992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.562+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9f0 tx=0x7f63f800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 shutdown_connections 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f63ec0a1a30 0x7f63ec0a3ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6408100bd0 0x7f64081a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408100230 0x7f6408199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 --2- 172.21.15.72:0/1223878365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64080ff8d0 0x7f64081992c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 >> 172.21.15.72:0/1223878365 conn(0x7f64080fb4f0 msgr2=0x7f64080fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 shutdown_connections 2024-03-11T19:19:38.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.563+0000 7f640ea00640 1 -- 172.21.15.72:0/1223878365 wait complete. 2024-03-11T19:19:38.586 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.780+0000 7fe357600640 1 -- 172.21.15.72:0/3026781432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3501033a0 msgr2=0x7fe3501037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.780+0000 7fe357600640 1 --2- 172.21.15.72:0/3026781432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3501033a0 0x7fe3501037c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe344009900 tx=0x7fe34402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 -- 172.21.15.72:0/3026781432 shutdown_connections 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 --2- 172.21.15.72:0/3026781432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe350104f50 0x7fe35010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 --2- 172.21.15.72:0/3026781432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104590 0x7fe350104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 --2- 172.21.15.72:0/3026781432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3501033a0 0x7fe3501037c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 -- 172.21.15.72:0/3026781432 >> 172.21.15.72:0/3026781432 conn(0x7fe3500fe930 msgr2=0x7fe350100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:38.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 -- 172.21.15.72:0/3026781432 shutdown_connections 2024-03-11T19:19:38.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.781+0000 7fe357600640 1 -- 172.21.15.72:0/3026781432 wait complete. 2024-03-11T19:19:38.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 Processor -- start 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 -- start start 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3501033a0 0x7fe350199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104f50 0x7fe3501a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501a0f00 con 0x7fe350104f50 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501a1070 con 0x7fe350104590 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe357600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501a1230 con 0x7fe3501033a0 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe354c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3501033a0 0x7fe350199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52202/0 (socket says 172.21.15.72:52202) 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe34fe00640 1 -- 172.21.15.72:0/3926986338 learned_addr learned my addr 172.21.15.72:0/3926986338 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.782+0000 7fe355600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104f50 0x7fe3501a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 -- 172.21.15.72:0/3926986338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3501033a0 msgr2=0x7fe350199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3501033a0 0x7fe350199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 -- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104f50 msgr2=0x7fe3501a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104f50 0x7fe3501a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 -- 172.21.15.72:0/3926986338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe344009560 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34fe00640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00e9f0 tx=0x7fe33c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c009700 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe33c00cd80 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3501a1510 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c018920 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.783+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3501a1a70 con 0x7fe350104590 2024-03-11T19:19:38.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.784+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35010a6c0 con 0x7fe350104590 2024-03-11T19:19:38.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.787+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe33c018a80 con 0x7fe350104590 2024-03-11T19:19:38.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.788+0000 7fe34d600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 0x7fe3380a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:38.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.788+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe33c014070 con 0x7fe350104590 2024-03-11T19:19:38.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.789+0000 7fe354c00640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 0x7fe3380a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:38.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.789+0000 7fe354c00640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 0x7fe3380a3f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe34402c240 tx=0x7fe34402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:38.792+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe33c052d20 con 0x7fe350104590 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1579249972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:38 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1223878365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1579249972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:38 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1223878365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:39.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.004+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe35002cda0 con 0x7fe350104590 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.005+0000 7fe34d600640 1 -- 172.21.15.72:0/3926986338 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe33c0969b0 con 0x7fe350104590 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:19:39.009 INFO:teuthology.orchestra.run.smithi072.stdout:] 2024-03-11T19:19:39.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 msgr2=0x7fe3380a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:39.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 0x7fe3380a3f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe34402c240 tx=0x7fe34402c7e0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 msgr2=0x7fe350199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00e9f0 tx=0x7fe33c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 shutdown_connections 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3380a1ab0 0x7fe3380a3f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe350104f50 0x7fe3501a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe350104590 0x7fe350199950 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 --2- 172.21.15.72:0/3926986338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3501033a0 0x7fe350199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 >> 172.21.15.72:0/3926986338 conn(0x7fe3500fe930 msgr2=0x7fe350107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 shutdown_connections 2024-03-11T19:19:39.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.007+0000 7fe357600640 1 -- 172.21.15.72:0/3926986338 wait complete. 2024-03-11T19:19:39.029 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-11T19:19:39.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1579249972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:39.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:38 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1223878365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:39.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 -- 172.21.15.72:0/2751802909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc901045b0 msgr2=0x7fcc90104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 --2- 172.21.15.72:0/2751802909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc901045b0 0x7fcc90104a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009900 tx=0x7fcc7c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 -- 172.21.15.72:0/2751802909 shutdown_connections 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 --2- 172.21.15.72:0/2751802909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc90104f70 0x7fcc9010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 --2- 172.21.15.72:0/2751802909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc901045b0 0x7fcc90104a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 --2- 172.21.15.72:0/2751802909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 0x7fcc901037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 -- 172.21.15.72:0/2751802909 >> 172.21.15.72:0/2751802909 conn(0x7fcc900fe930 msgr2=0x7fcc90100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 -- 172.21.15.72:0/2751802909 shutdown_connections 2024-03-11T19:19:39.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.229+0000 7fcc97400640 1 -- 172.21.15.72:0/2751802909 wait complete. 2024-03-11T19:19:39.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.230+0000 7fcc97400640 1 Processor -- start 2024-03-11T19:19:39.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.230+0000 7fcc97400640 1 -- start start 2024-03-11T19:19:39.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 0x7fcc90199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc90104f70 0x7fcc901a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901a0f60 con 0x7fcc90104f70 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901a10d0 con 0x7fcc901045b0 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc97400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901a1290 con 0x7fcc901033c0 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52218/0 (socket says 172.21.15.72:52218) 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 -- 172.21.15.72:0/604227445 learned_addr learned my addr 172.21.15.72:0/604227445 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc94a00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 0x7fcc90199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc95400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc90104f70 0x7fcc901a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 -- 172.21.15.72:0/604227445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 msgr2=0x7fcc90199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 0x7fcc90199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 -- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc90104f70 msgr2=0x7fcc901a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:39.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc90104f70 0x7fcc901a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.231+0000 7fcc8fe00640 1 -- 172.21.15.72:0/604227445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c009560 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc8fe00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c6a0 tx=0x7fcc7c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0044c0 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc901a1510 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c0052d0 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c03e450 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.232+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc901a1a20 con 0x7fcc901045b0 2024-03-11T19:19:39.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.233+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9010a760 con 0x7fcc901045b0 2024-03-11T19:19:39.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.235+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcc7c005440 con 0x7fcc901045b0 2024-03-11T19:19:39.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.237+0000 7fcc8d600640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 0x7fcc780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:39.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.237+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcc7c0ef850 con 0x7fcc901045b0 2024-03-11T19:19:39.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.237+0000 7fcc94a00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 0x7fcc780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:39.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.238+0000 7fcc94a00640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 0x7fcc780a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc90104410 tx=0x7fcc84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:39.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.240+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc7c0b8e60 con 0x7fcc901045b0 2024-03-11T19:19:39.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:39.448+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc9010ee20 con 0x7fcc901045b0 2024-03-11T19:19:40.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.374+0000 7fcc8d600640 1 -- 172.21.15.72:0/604227445 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc7c0bc520 con 0x7fcc901045b0 2024-03-11T19:19:40.379 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2024-03-11T19:19:40.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.380+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 msgr2=0x7fcc780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:40.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.380+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 0x7fcc780a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc90104410 tx=0x7fcc84005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:40.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.380+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 msgr2=0x7fcc901999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:40.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.380+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c6a0 tx=0x7fcc7c0040c0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 shutdown_connections 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcc780a1a30 0x7fcc780a3ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc90104f70 0x7fcc901a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc901045b0 0x7fcc901999b0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c6a0 tx=0x7fcc7c0040c0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 --2- 172.21.15.72:0/604227445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc901033c0 0x7fcc90199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.381+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 >> 172.21.15.72:0/604227445 conn(0x7fcc900fe930 msgr2=0x7fcc90107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:40.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.382+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 shutdown_connections 2024-03-11T19:19:40.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.383+0000 7fcc97400640 1 -- 172.21.15.72:0/604227445 wait complete. 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3926986338' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:40.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:39 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3926986338' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:40.404 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:39 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.406 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3926986338' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:40.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:39 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 -- 172.21.15.72:0/1399776854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0102380 msgr2=0x7f79f0102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 --2- 172.21.15.72:0/1399776854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0102380 0x7f79f0102800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009900 tx=0x7f79dc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 -- 172.21.15.72:0/1399776854 shutdown_connections 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 --2- 172.21.15.72:0/1399776854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 0x7f79f01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 --2- 172.21.15.72:0/1399776854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0102380 0x7f79f0102800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 --2- 172.21.15.72:0/1399776854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0101190 0x7f79f01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 -- 172.21.15.72:0/1399776854 >> 172.21.15.72:0/1399776854 conn(0x7f79f00fc700 msgr2=0x7f79f00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 -- 172.21.15.72:0/1399776854 shutdown_connections 2024-03-11T19:19:40.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.594+0000 7f79f5200640 1 -- 172.21.15.72:0/1399776854 wait complete. 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.595+0000 7f79f5200640 1 Processor -- start 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 -- start start 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 0x7f79f01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 0x7f79f019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019ed50 con 0x7f79f0102380 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019eec0 con 0x7f79f0101190 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79f5200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f080 con 0x7f79f0102d40 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79ede00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79ed400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 0x7f79f01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79ede00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:52240/0 (socket says 172.21.15.72:52240) 2024-03-11T19:19:40.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79ede00640 1 -- 172.21.15.72:0/1075897946 learned_addr learned my addr 172.21.15.72:0/1075897946 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.596+0000 7f79ee800640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 0x7f79f019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 -- 172.21.15.72:0/1075897946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 msgr2=0x7f79f019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 0x7f79f019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 -- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 msgr2=0x7f79f01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 0x7f79f01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 -- 172.21.15.72:0/1075897946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009560 con 0x7f79f0101190 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ed400640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 0x7f79f01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79ede00640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9c0 tx=0x7f79e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0009700 con 0x7f79f0101190 2024-03-11T19:19:40.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79e000cd80 con 0x7f79f0101190 2024-03-11T19:19:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f019f360 con 0x7f79f0101190 2024-03-11T19:19:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0010430 con 0x7f79f0101190 2024-03-11T19:19:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.597+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f019f840 con 0x7f79f0101190 2024-03-11T19:19:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.599+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f01084b0 con 0x7f79f0101190 2024-03-11T19:19:40.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.600+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f79e0003680 con 0x7f79f0101190 2024-03-11T19:19:40.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.602+0000 7f79e6a00640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 0x7f79d40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:40.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.603+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f79e0014070 con 0x7f79f0101190 2024-03-11T19:19:40.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.603+0000 7f79ed400640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 0x7f79d40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:40.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.603+0000 7f79ed400640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 0x7f79d40a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:40.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.605+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79e00969c0 con 0x7f79f0101190 2024-03-11T19:19:40.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:40.813+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f79f010cbf0 con 0x7f79f0101190 2024-03-11T19:19:41.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.376+0000 7f79e6a00640 1 -- 172.21.15.72:0/1075897946 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7f79e009a080 con 0x7f79f0101190 2024-03-11T19:19:41.381 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2024-03-11T19:19:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.384+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 msgr2=0x7f79d40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.384+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 0x7f79d40a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.384+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 msgr2=0x7f79f0197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.384+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9c0 tx=0x7f79e000ee80 comp rx=0 tx=0).stop 2024-03-11T19:19:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 shutdown_connections 2024-03-11T19:19:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f79d40a1a30 0x7f79d40a3ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0102d40 0x7f79f019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79f0102380 0x7f79f01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 --2- 172.21.15.72:0/1075897946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79f0101190 0x7f79f0197260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.386+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 >> 172.21.15.72:0/1075897946 conn(0x7f79f00fc700 msgr2=0x7f79f0105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.387+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 shutdown_connections 2024-03-11T19:19:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.387+0000 7f79f5200640 1 -- 172.21.15.72:0/1075897946 wait complete. 2024-03-11T19:19:41.419 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/604227445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:41.475 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:19:41.476 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:19:41.476 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:41 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1075897946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:41.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 -- 172.21.15.72:0/2202232382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64101060 msgr2=0x7f9d64101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:41.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 --2- 172.21.15.72:0/2202232382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64101060 0x7f9d64101480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009900 tx=0x7f9d5402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:41.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 -- 172.21.15.72:0/2202232382 shutdown_connections 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 --2- 172.21.15.72:0/2202232382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64102c10 0x7f9d641094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 --2- 172.21.15.72:0/2202232382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d641026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 --2- 172.21.15.72:0/2202232382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64101060 0x7f9d64101480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 -- 172.21.15.72:0/2202232382 >> 172.21.15.72:0/2202232382 conn(0x7f9d640fc5f0 msgr2=0x7f9d640fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 -- 172.21.15.72:0/2202232382 shutdown_connections 2024-03-11T19:19:41.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.591+0000 7f9d63e00640 1 -- 172.21.15.72:0/2202232382 wait complete. 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 Processor -- start 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 -- start start 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d64197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64102c10 0x7f9d6419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419ebd0 con 0x7f9d64102c10 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419ed40 con 0x7f9d64102250 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419ef00 con 0x7f9d64101060 2024-03-11T19:19:41.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.592+0000 7f9d63400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64102c10 0x7f9d6419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:53446/0 (socket says 172.21.15.72:53446) 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 -- 172.21.15.72:0/606942286 learned_addr learned my addr 172.21.15.72:0/606942286 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d5be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d64197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 -- 172.21.15.72:0/606942286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 msgr2=0x7f9d64197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d64197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 -- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64102c10 msgr2=0x7f9d6419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64102c10 0x7f9d6419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 -- 172.21.15.72:0/606942286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009490 con 0x7f9d64101060 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d5be00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d64197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d62a00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005ab0 tx=0x7f9d54005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.593+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540043d0 con 0x7f9d64101060 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.594+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d540052d0 con 0x7f9d64101060 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.594+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009560 con 0x7f9d64101060 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.594+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54003d40 con 0x7f9d64101060 2024-03-11T19:19:41.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.594+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6419f500 con 0x7f9d64101060 2024-03-11T19:19:41.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.595+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d64108380 con 0x7f9d64101060 2024-03-11T19:19:41.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.596+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9d54005440 con 0x7f9d64101060 2024-03-11T19:19:41.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.598+0000 7f9d5b400640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 0x7f9d440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:41.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.598+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9d540ef6c0 con 0x7f9d64101060 2024-03-11T19:19:41.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.598+0000 7f9d5be00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 0x7f9d440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:41.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.598+0000 7f9d5be00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 0x7f9d440a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d641985f0 tx=0x7f9d4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:41.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.600+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d540b89e0 con 0x7f9d64101060 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/604227445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:19:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:41 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1075897946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:41.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/604227445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:19:41.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:19:41.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:41.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:41.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:19:41.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:19:41.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:41 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1075897946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:41.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:41.833+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9d6402ce10 con 0x7f9d64101060 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/606942286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:19:42.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:42 smithi106 ceph-mon[35889]: pgmap v320: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/606942286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:19:42.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:42 smithi072 ceph-mon[31695]: pgmap v320: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:42.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/606942286' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:19:42.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:19:42.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:42 smithi143 ceph-mon[36013]: pgmap v320: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:42.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.752+0000 7f9d5b400640 1 -- 172.21.15.72:0/606942286 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9d540bc0a0 con 0x7f9d64101060 2024-03-11T19:19:42.757 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 2 and data pool 3 2024-03-11T19:19:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.758+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 msgr2=0x7f9d440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.758+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 0x7f9d440a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d641985f0 tx=0x7f9d4c005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.758+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 msgr2=0x7f9d641970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:42.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.758+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005ab0 tx=0x7f9d54005e50 comp rx=0 tx=0).stop 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 shutdown_connections 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9d440a1a30 0x7f9d440a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d64102c10 0x7f9d6419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d64102250 0x7f9d64197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 --2- 172.21.15.72:0/606942286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d64101060 0x7f9d641970e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005ab0 tx=0x7f9d54005e50 comp rx=0 tx=0).stop 2024-03-11T19:19:42.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.759+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 >> 172.21.15.72:0/606942286 conn(0x7f9d640fc5f0 msgr2=0x7f9d64105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:42.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.761+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 shutdown_connections 2024-03-11T19:19:42.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.761+0000 7f9d63e00640 1 -- 172.21.15.72:0/606942286 wait complete. 2024-03-11T19:19:42.785 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-11T19:19:42.785 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-11T19:19:42.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 -- 172.21.15.72:0/4274345628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc108990 msgr2=0x7f21bc108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 --2- 172.21.15.72:0/4274345628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc108990 0x7f21bc108e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009900 tx=0x7f21a402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 -- 172.21.15.72:0/4274345628 shutdown_connections 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 --2- 172.21.15.72:0/4274345628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc109350 0x7f21bc10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 --2- 172.21.15.72:0/4274345628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc108990 0x7f21bc108e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 --2- 172.21.15.72:0/4274345628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc1077a0 0x7f21bc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 -- 172.21.15.72:0/4274345628 >> 172.21.15.72:0/4274345628 conn(0x7f21bc076740 msgr2=0x7f21bc078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 -- 172.21.15.72:0/4274345628 shutdown_connections 2024-03-11T19:19:42.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.977+0000 7f21c0c00640 1 -- 172.21.15.72:0/4274345628 wait complete. 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.978+0000 7f21c0c00640 1 Processor -- start 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 -- start start 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 0x7f21bc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc109350 0x7f21bc1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1a5360 con 0x7f21bc1077a0 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1a54d0 con 0x7f21bc108990 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21c0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1a5690 con 0x7f21bc109350 2024-03-11T19:19:42.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54820/0 (socket says 172.21.15.72:54820) 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21b9e00640 1 -- 172.21.15.72:0/179941533 learned_addr learned my addr 172.21.15.72:0/179941533 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.979+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 0x7f21bc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21ba800640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc109350 0x7f21bc1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 -- 172.21.15.72:0/179941533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc109350 msgr2=0x7f21bc1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc109350 0x7f21bc1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 -- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 msgr2=0x7f21bc19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 -- 172.21.15.72:0/179941533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009560 con 0x7f21bc108990 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b9e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:42.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b3e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 0x7f21bc19ddb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000c00 tx=0x7f21a4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.980+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4004aa0 con 0x7f21bc108990 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.981+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a40052d0 con 0x7f21bc108990 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.981+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc1a5910 con 0x7f21bc108990 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.981+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003e80 con 0x7f21bc108990 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.981+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc1a5df0 con 0x7f21bc108990 2024-03-11T19:19:42.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.982+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc10eae0 con 0x7f21bc108990 2024-03-11T19:19:42.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.984+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f21a4005440 con 0x7f21bc108990 2024-03-11T19:19:42.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.985+0000 7f21b2a00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 0x7f21a00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:42.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.985+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21a40ef990 con 0x7f21bc108990 2024-03-11T19:19:42.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.985+0000 7f21b9e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 0x7f21a00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:42.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.986+0000 7f21b9e00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 0x7f21a00a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0058c0 tx=0x7f21ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:42.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:42.987+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21a40b8c50 con 0x7f21bc108990 2024-03-11T19:19:43.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.195+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f21bc113390 con 0x7f21bc108990 2024-03-11T19:19:43.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.361+0000 7f21b2a00640 1 -- 172.21.15.72:0/179941533 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7f21a40bc310 con 0x7f21bc108990 2024-03-11T19:19:43.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 msgr2=0x7f21a00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:43.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 0x7f21a00a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0058c0 tx=0x7f21ac005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 msgr2=0x7f21bc19ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 0x7f21bc19ddb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000c00 tx=0x7f21a4004270 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 shutdown_connections 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f21a00a1a30 0x7f21a00a3ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc109350 0x7f21bc1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21bc108990 0x7f21bc19ddb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 --2- 172.21.15.72:0/179941533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21bc1077a0 0x7f21bc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 >> 172.21.15.72:0/179941533 conn(0x7f21bc076740 msgr2=0x7f21bc10bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 shutdown_connections 2024-03-11T19:19:43.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.366+0000 7f21c0c00640 1 -- 172.21.15.72:0/179941533 wait complete. 2024-03-11T19:19:43.391 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:43.602 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/179941533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.603 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:43 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 -- 172.21.15.72:0/1244859174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94104570 msgr2=0x7fbe94106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/1244859174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94104570 0x7fbe94106960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009900 tx=0x7fbe8002bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 -- 172.21.15.72:0/1244859174 shutdown_connections 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/1244859174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94106ea0 0x7fbe941092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/1244859174 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94104570 0x7fbe94106960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/1244859174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe94104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 -- 172.21.15.72:0/1244859174 >> 172.21.15.72:0/1244859174 conn(0x7fbe940fb630 msgr2=0x7fbe940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 -- 172.21.15.72:0/1244859174 shutdown_connections 2024-03-11T19:19:43.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.608+0000 7fbe9ac00640 1 -- 172.21.15.72:0/1244859174 wait complete. 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 Processor -- start 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 -- start start 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 0x7fbe941999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94106ea0 0x7fbe941a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a0fa0 con 0x7fbe94106ea0 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a1110 con 0x7fbe94104570 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.609+0000 7fbe9ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a12d0 con 0x7fbe94101c40 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:43.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 0x7fbe941999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56746/0 (socket says 172.21.15.72:56746) 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93e00640 1 -- 172.21.15.72:0/4130970622 learned_addr learned my addr 172.21.15.72:0/4130970622 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe98c00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94106ea0 0x7fbe941a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 -- 172.21.15.72:0/4130970622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 msgr2=0x7fbe941994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 -- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94106ea0 msgr2=0x7fbe941a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94106ea0 0x7fbe941a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 -- 172.21.15.72:0/4130970622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80009560 con 0x7fbe94104570 2024-03-11T19:19:43.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93e00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.610+0000 7fbe93400640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 0x7fbe941999f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe80036040 tx=0x7fbe80004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.611+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80004aa0 con 0x7fbe94104570 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.611+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe941a1550 con 0x7fbe94104570 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.611+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe800052d0 con 0x7fbe94104570 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.611+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80003dc0 con 0x7fbe94104570 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.611+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a1a80 con 0x7fbe94104570 2024-03-11T19:19:43.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.612+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe941089f0 con 0x7fbe94104570 2024-03-11T19:19:43.618 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.614+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbe80005440 con 0x7fbe94104570 2024-03-11T19:19:43.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.615+0000 7fbe90c00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 0x7fbe780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:43.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.615+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fbe800efea0 con 0x7fbe94104570 2024-03-11T19:19:43.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.615+0000 7fbe93e00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 0x7fbe780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:43.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.616+0000 7fbe93e00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 0x7fbe780a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011500 tx=0x7fbe84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:43.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.617+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe800b9020 con 0x7fbe94104570 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:43.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:19:43.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:19:43.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:19:43.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/179941533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:43 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:19:43.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/179941533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:43 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:19:43.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:43.818+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe9410cdd0 con 0x7fbe94104570 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:active 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/4130970622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:44 smithi106 ceph-mon[35889]: pgmap v323: 97 pgs: 7 creating+peering, 35 active+clean, 55 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:active 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4130970622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:44 smithi072 ceph-mon[31695]: pgmap v323: 97 pgs: 7 creating+peering, 35 active+clean, 55 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-11T19:19:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:19:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:19:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] up:active 2024-03-11T19:19:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:44.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/4130970622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:19:44.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:44 smithi143 ceph-mon[36013]: pgmap v323: 97 pgs: 7 creating+peering, 35 active+clean, 55 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-11T19:19:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:45 smithi106 ceph-mon[35889]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-11T19:19:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:45 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:19:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:45 smithi106 ceph-mon[35889]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:19:45.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:45 smithi072 ceph-mon[31695]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-11T19:19:45.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:45 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:19:45.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:45 smithi072 ceph-mon[31695]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:19:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:45 smithi143 ceph-mon[36013]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-11T19:19:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:45 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:19:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:45 smithi143 ceph-mon[36013]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:19:45.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.776+0000 7fbe90c00640 1 -- 172.21.15.72:0/4130970622 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbe800bc6e0 con 0x7fbe94104570 2024-03-11T19:19:45.781 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data_ec' created 2024-03-11T19:19:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.782+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 msgr2=0x7fbe780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.782+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 0x7fbe780a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe84011500 tx=0x7fbe84005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.782+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 msgr2=0x7fbe941999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:45.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.782+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 0x7fbe941999f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe80036040 tx=0x7fbe80004270 comp rx=0 tx=0).stop 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 shutdown_connections 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbe780a1a30 0x7fbe780a3ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe94106ea0 0x7fbe941a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe94104570 0x7fbe941999f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 --2- 172.21.15.72:0/4130970622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe94101c40 0x7fbe941994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.784+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 >> 172.21.15.72:0/4130970622 conn(0x7fbe940fb630 msgr2=0x7fbe94101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:45.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.785+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 shutdown_connections 2024-03-11T19:19:45.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.785+0000 7fbe9ac00640 1 -- 172.21.15.72:0/4130970622 wait complete. 2024-03-11T19:19:45.814 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 -- 172.21.15.72:0/2264736604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea681089b0 msgr2=0x7fea68108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 --2- 172.21.15.72:0/2264736604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea681089b0 0x7fea68108e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea58009900 tx=0x7fea5802bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 -- 172.21.15.72:0/2264736604 shutdown_connections 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 --2- 172.21.15.72:0/2264736604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea68109370 0x7fea6810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 --2- 172.21.15.72:0/2264736604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea681089b0 0x7fea68108e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 --2- 172.21.15.72:0/2264736604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681077c0 0x7fea68107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 -- 172.21.15.72:0/2264736604 >> 172.21.15.72:0/2264736604 conn(0x7fea68076770 msgr2=0x7fea68078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 -- 172.21.15.72:0/2264736604 shutdown_connections 2024-03-11T19:19:45.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.994+0000 7fea6fa00640 1 -- 172.21.15.72:0/2264736604 wait complete. 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.995+0000 7fea6fa00640 1 Processor -- start 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 -- start start 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 0x7fea6819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 0x7fea6819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea681a5320 con 0x7fea681089b0 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea681a5490 con 0x7fea681077c0 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6fa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea681a5650 con 0x7fea68109370 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea67e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 0x7fea6819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 0x7fea6819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:46.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56766/0 (socket says 172.21.15.72:56766) 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea67e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 0x7fea6819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54860/0 (socket says 172.21.15.72:54860) 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.996+0000 7fea6da00640 1 -- 172.21.15.72:0/1914766181 learned_addr learned my addr 172.21.15.72:0/1914766181 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 -- 172.21.15.72:0/1914766181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 msgr2=0x7fea6819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 0x7fea6819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 -- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 msgr2=0x7fea6819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 0x7fea6819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 -- 172.21.15.72:0/1914766181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009560 con 0x7fea68109370 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6d000640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 0x7fea6819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6da00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea6000eb10 tx=0x7fea6000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60004b20 con 0x7fea68109370 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea60004c80 con 0x7fea68109370 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea681a5930 con 0x7fea68109370 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60010670 con 0x7fea68109370 2024-03-11T19:19:46.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.997+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea681a5e10 con 0x7fea68109370 2024-03-11T19:19:46.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:45.998+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6810ea80 con 0x7fea68109370 2024-03-11T19:19:46.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.000+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fea60004df0 con 0x7fea68109370 2024-03-11T19:19:46.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.002+0000 7fea65600640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 0x7fea500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:46.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.002+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fea60014070 con 0x7fea68109370 2024-03-11T19:19:46.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.002+0000 7fea6d000640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 0x7fea500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:46.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.002+0000 7fea6d000640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 0x7fea500a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea68108810 tx=0x7fea5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:46.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.004+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea60096d90 con 0x7fea68109370 2024-03-11T19:19:46.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.200+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fea681133f0 con 0x7fea68109370 2024-03-11T19:19:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.788+0000 7fea65600640 1 -- 172.21.15.72:0/1914766181 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7fea6009a450 con 0x7fea68109370 2024-03-11T19:19:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:set pool 4 allow_ec_overwrites to true 2024-03-11T19:19:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.794+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 msgr2=0x7fea500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.794+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 0x7fea500a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea68108810 tx=0x7fea5c005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.794+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 msgr2=0x7fea681a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:46.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.794+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea6000eb10 tx=0x7fea6000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 shutdown_connections 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fea500a1a30 0x7fea500a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68109370 0x7fea681a4de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea681089b0 0x7fea6819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 --2- 172.21.15.72:0/1914766181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea681077c0 0x7fea6819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 >> 172.21.15.72:0/1914766181 conn(0x7fea68076770 msgr2=0x7fea6810bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:46.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 shutdown_connections 2024-03-11T19:19:46.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:46.795+0000 7fea6fa00640 1 -- 172.21.15.72:0/1914766181 wait complete. 2024-03-11T19:19:46.836 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 -- 172.21.15.72:0/2872801505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3901076a0 msgr2=0x7fe390107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 --2- 172.21.15.72:0/2872801505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3901076a0 0x7fe390107ac0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe380009900 tx=0x7fe38002bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 -- 172.21.15.72:0/2872801505 shutdown_connections 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 --2- 172.21.15.72:0/2872801505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 0x7fe39010fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 --2- 172.21.15.72:0/2872801505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe390108890 0x7fe390108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 --2- 172.21.15.72:0/2872801505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3901076a0 0x7fe390107ac0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 -- 172.21.15.72:0/2872801505 >> 172.21.15.72:0/2872801505 conn(0x7fe390076650 msgr2=0x7fe390078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 -- 172.21.15.72:0/2872801505 shutdown_connections 2024-03-11T19:19:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.018+0000 7fe38fe00640 1 -- 172.21.15.72:0/2872801505 wait complete. 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 Processor -- start 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 -- start start 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3901076a0 0x7fe39019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 0x7fe3901a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901a51f0 con 0x7fe390108890 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901a5360 con 0x7fe3901076a0 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901a5520 con 0x7fe390109250 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54870/0 (socket says 172.21.15.72:54870) 2024-03-11T19:19:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3901076a0 0x7fe39019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38e000640 1 -- 172.21.15.72:0/974574318 learned_addr learned my addr 172.21.15.72:0/974574318 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.019+0000 7fe38f400640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 0x7fe3901a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 -- 172.21.15.72:0/974574318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 msgr2=0x7fe3901a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 0x7fe3901a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 -- 172.21.15.72:0/974574318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3901076a0 msgr2=0x7fe39019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3901076a0 0x7fe39019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 -- 172.21.15.72:0/974574318 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009560 con 0x7fe390108890 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38e000640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e9f0 tx=0x7fe37c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009700 con 0x7fe390108890 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3901a5800 con 0x7fe390108890 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37c00cd80 con 0x7fe390108890 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.020+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c018920 con 0x7fe390108890 2024-03-11T19:19:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.021+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3901a5d60 con 0x7fe390108890 2024-03-11T19:19:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.022+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39010e9c0 con 0x7fe390108890 2024-03-11T19:19:47.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.024+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe37c018a80 con 0x7fe390108890 2024-03-11T19:19:47.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.026+0000 7fe387400640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 0x7fe3740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.027+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fe37c014070 con 0x7fe390108890 2024-03-11T19:19:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.027+0000 7fe38ea00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 0x7fe3740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:47.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.027+0000 7fe38ea00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 0x7fe3740a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe380005ab0 tx=0x7fe380005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:47.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.029+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe37c096ed0 con 0x7fe390108890 2024-03-11T19:19:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:46 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:19:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:46 smithi106 ceph-mon[35889]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:19:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:46 smithi106 ceph-mon[35889]: pgmap v326: 129 pgs: 7 creating+peering, 84 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-03-11T19:19:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:46 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1914766181' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:46 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:46 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:19:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:46 smithi072 ceph-mon[31695]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:19:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:46 smithi072 ceph-mon[31695]: pgmap v326: 129 pgs: 7 creating+peering, 84 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-03-11T19:19:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1914766181' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:46 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:46 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:19:47.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:46 smithi143 ceph-mon[36013]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:19:47.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:46 smithi143 ceph-mon[36013]: pgmap v326: 129 pgs: 7 creating+peering, 84 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-03-11T19:19:47.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:46 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1914766181' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:46 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:19:47.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.261+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe3901130b0 con 0x7fe390108890 2024-03-11T19:19:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.784+0000 7fe387400640 1 -- 172.21.15.72:0/974574318 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe37c09a590 con 0x7fe390108890 2024-03-11T19:19:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:added data pool 4 to fsmap 2024-03-11T19:19:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 msgr2=0x7fe3740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 0x7fe3740a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe380005ab0 tx=0x7fe380005fd0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 msgr2=0x7fe39019dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e9f0 tx=0x7fe37c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 shutdown_connections 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe3740a1a30 0x7fe3740a3ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe390109250 0x7fe3901a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108890 0x7fe39019dc40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 --2- 172.21.15.72:0/974574318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3901076a0 0x7fe39019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 >> 172.21.15.72:0/974574318 conn(0x7fe390076650 msgr2=0x7fe39010bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 shutdown_connections 2024-03-11T19:19:47.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:47.788+0000 7fe38fe00640 1 -- 172.21.15.72:0/974574318 wait complete. 2024-03-11T19:19:47.815 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:48.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 -- 172.21.15.72:0/2023316802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 msgr2=0x7f61a0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 --2- 172.21.15.72:0/2023316802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0104a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6188009900 tx=0x7f618802bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 -- 172.21.15.72:0/2023316802 shutdown_connections 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 --2- 172.21.15.72:0/2023316802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a0104f70 0x7f61a010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 --2- 172.21.15.72:0/2023316802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0104a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 --2- 172.21.15.72:0/2023316802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a01033c0 0x7f61a01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 -- 172.21.15.72:0/2023316802 >> 172.21.15.72:0/2023316802 conn(0x7f61a00fe970 msgr2=0x7f61a0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 -- 172.21.15.72:0/2023316802 shutdown_connections 2024-03-11T19:19:48.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.013+0000 7f61a4a00640 1 -- 172.21.15.72:0/2023316802 wait complete. 2024-03-11T19:19:48.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.014+0000 7f61a4a00640 1 Processor -- start 2024-03-11T19:19:48.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.014+0000 7f61a4a00640 1 -- start start 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.014+0000 7f61a4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a01033c0 0x7f61a0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f61a4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f61a4a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 0x7f61a01a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f61a4a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01a0f10 con 0x7f61a01045b0 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f61a4a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01a1080 con 0x7f61a0104f70 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f61a4a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01a1240 con 0x7f61a01033c0 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54884/0 (socket says 172.21.15.72:54884) 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 -- 172.21.15.72:0/323680272 learned_addr learned my addr 172.21.15.72:0/323680272 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f619de00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a01033c0 0x7f61a0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f619e800640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 0x7f61a01a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 -- 172.21.15.72:0/323680272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a01033c0 msgr2=0x7f61a0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a01033c0 0x7f61a0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 -- 172.21.15.72:0/323680272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 msgr2=0x7f61a01a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 0x7f61a01a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 -- 172.21.15.72:0/323680272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188009560 con 0x7f61a01045b0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f619e800640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 0x7f61a01a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.015+0000 7f6197e00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6188006010 tx=0x7f6188005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.016+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61880043d0 con 0x7f61a01045b0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.016+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61880052d0 con 0x7f61a01045b0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.016+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188003cc0 con 0x7f61a01045b0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.016+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a01a14c0 con 0x7f61a01045b0 2024-03-11T19:19:48.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.016+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01a19d0 con 0x7f61a01045b0 2024-03-11T19:19:48.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.017+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a010a760 con 0x7f61a01045b0 2024-03-11T19:19:48.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.019+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6188005440 con 0x7f61a01045b0 2024-03-11T19:19:48.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.020+0000 7f6196a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 0x7f61800a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.020+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f61880ef910 con 0x7f61a01045b0 2024-03-11T19:19:48.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.020+0000 7f619de00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 0x7f61800a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.021+0000 7f619de00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 0x7f61800a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f61a0104410 tx=0x7f6190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.024+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61880b88e0 con 0x7f61a01045b0 2024-03-11T19:19:48.145 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:19:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: 2024-03-11T19:19:47.684+0000 7fe59fda4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-11T19:19:48.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:47 smithi106 ceph-mon[35889]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:48.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:47 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:19:48.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:47 smithi106 ceph-mon[35889]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:19:48.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:47 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:19:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:47 smithi072 ceph-mon[31695]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:47 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:19:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:47 smithi072 ceph-mon[31695]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:19:48.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:47 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:19:48.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:47 smithi143 ceph-mon[36013]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:19:48.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:47 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:19:48.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:47 smithi143 ceph-mon[36013]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:19:48.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:47 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:19:48.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.262+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61a002ce40 con 0x7f61a01045b0 2024-03-11T19:19:48.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.264+0000 7f6196a00640 1 -- 172.21.15.72:0/323680272 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11286 (secure 0 0 0) 0x7f61880bbfa0 con 0x7f61a01045b0 2024-03-11T19:19:48.268 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:48.268 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:19:47.263762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T19:19:48.268 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 17 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.266+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 msgr2=0x7f61800a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 0x7f61800a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f61a0104410 tx=0x7f6190005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 msgr2=0x7f61a0199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6188006010 tx=0x7f6188005e80 comp rx=0 tx=0).stop 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 shutdown_connections 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f61800a1a30 0x7f61800a3ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61a0104f70 0x7f61a01a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a01045b0 0x7f61a0199960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 --2- 172.21.15.72:0/323680272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61a01033c0 0x7f61a0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 >> 172.21.15.72:0/323680272 conn(0x7f61a00fe970 msgr2=0x7f61a0107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:48.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 shutdown_connections 2024-03-11T19:19:48.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.267+0000 7f61a4a00640 1 -- 172.21.15.72:0/323680272 wait complete. 2024-03-11T19:19:48.290 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.492+0000 7fcb14000640 1 -- 172.21.15.72:0/960020353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 msgr2=0x7fcb0c0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.492+0000 7fcb14000640 1 --2- 172.21.15.72:0/960020353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c0771e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009900 tx=0x7fcaf802bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 -- 172.21.15.72:0/960020353 shutdown_connections 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 --2- 172.21.15.72:0/960020353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 0x7fcb0c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 --2- 172.21.15.72:0/960020353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c0771e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 --2- 172.21.15.72:0/960020353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 -- 172.21.15.72:0/960020353 >> 172.21.15.72:0/960020353 conn(0x7fcb0c0fe930 msgr2=0x7fcb0c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 -- 172.21.15.72:0/960020353 shutdown_connections 2024-03-11T19:19:48.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.493+0000 7fcb14000640 1 -- 172.21.15.72:0/960020353 wait complete. 2024-03-11T19:19:48.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 Processor -- start 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 -- start start 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 0x7fcb0c1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a5310 con 0x7fcb0c076d80 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a5480 con 0x7fcb0c077720 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb14000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a5640 con 0x7fcb0c075b40 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb10c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb11600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb12000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 0x7fcb0c1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb10c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54900/0 (socket says 172.21.15.72:54900) 2024-03-11T19:19:48.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.494+0000 7fcb10c00640 1 -- 172.21.15.72:0/3358406785 learned_addr learned my addr 172.21.15.72:0/3358406785 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 -- 172.21.15.72:0/3358406785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 msgr2=0x7fcb0c1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 0x7fcb0c1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 -- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 msgr2=0x7fcb0c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 -- 172.21.15.72:0/3358406785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8009560 con 0x7fcb0c075b40 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb11600640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c19d820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e9f0 tx=0x7fcb0000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000ccc0 con 0x7fcb0c075b40 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb0000ce20 con 0x7fcb0c075b40 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1a5920 con 0x7fcb0c075b40 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.495+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00004a00 con 0x7fcb0c075b40 2024-03-11T19:19:48.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.496+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1a5de0 con 0x7fcb0c075b40 2024-03-11T19:19:48.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.497+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c10fe80 con 0x7fcb0c075b40 2024-03-11T19:19:48.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.499+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcb000050b0 con 0x7fcb0c075b40 2024-03-11T19:19:48.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.500+0000 7fcb0a000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 0x7fcaf40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.500+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcb00014070 con 0x7fcb0c075b40 2024-03-11T19:19:48.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.500+0000 7fcb10c00640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 0x7fcaf40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.501+0000 7fcb10c00640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 0x7fcaf40a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c19ed30 tx=0x7fcaf802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.502+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb00096540 con 0x7fcb0c075b40 2024-03-11T19:19:48.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.686+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb0c113210 con 0x7fcb0c075b40 2024-03-11T19:19:48.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.689+0000 7fcb0a000640 1 -- 172.21.15.72:0/3358406785 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7fcb00099c00 con 0x7fcb0c075b40 2024-03-11T19:19:48.693 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:48.694 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.691+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 msgr2=0x7fcaf40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.691+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 0x7fcaf40a3ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c19ed30 tx=0x7fcaf802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 msgr2=0x7fcb0c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c19d820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e9f0 tx=0x7fcb0000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 shutdown_connections 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcaf40a1a30 0x7fcaf40a3ef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb0c077720 0x7fcb0c1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb0c076d80 0x7fcb0c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 --2- 172.21.15.72:0/3358406785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb0c075b40 0x7fcb0c19d820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 >> 172.21.15.72:0/3358406785 conn(0x7fcb0c0fe930 msgr2=0x7fcb0c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:48.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 shutdown_connections 2024-03-11T19:19:48.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.692+0000 7fcb14000640 1 -- 172.21.15.72:0/3358406785 wait complete. 2024-03-11T19:19:48.715 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 -- 172.21.15.72:0/577379327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f41045b0 msgr2=0x7fd7f4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/577379327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f41045b0 0x7fd7f4104a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009900 tx=0x7fd7e002bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 -- 172.21.15.72:0/577379327 shutdown_connections 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/577379327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104f70 0x7fd7f410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/577379327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f41045b0 0x7fd7f4104a30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/577379327 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 -- 172.21.15.72:0/577379327 >> 172.21.15.72:0/577379327 conn(0x7fd7f40fe970 msgr2=0x7fd7f4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 -- 172.21.15.72:0/577379327 shutdown_connections 2024-03-11T19:19:48.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.921+0000 7fd7fbe00640 1 -- 172.21.15.72:0/577379327 wait complete. 2024-03-11T19:19:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.922+0000 7fd7fbe00640 1 Processor -- start 2024-03-11T19:19:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.922+0000 7fd7fbe00640 1 -- start start 2024-03-11T19:19:48.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f4104f70 0x7fd7f41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a0f40 con 0x7fd7f41033c0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a10b0 con 0x7fd7f41045b0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7fbe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f41a1270 con 0x7fd7f4104f70 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:39946/0 (socket says 172.21.15.72:39946) 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 -- 172.21.15.72:0/1349821981 learned_addr learned my addr 172.21.15.72:0/1349821981 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f9e00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f4104f70 0x7fd7f41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 -- 172.21.15.72:0/1349821981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f4104f70 msgr2=0x7fd7f41a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f4104f70 0x7fd7f41a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 -- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 msgr2=0x7fd7f4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f8a00640 1 -- 172.21.15.72:0/1349821981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0009560 con 0x7fd7f41045b0 2024-03-11T19:19:48.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.923+0000 7fd7f9400640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7f8a00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006010 tx=0x7fd7e00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00044c0 con 0x7fd7f41045b0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f41a14f0 con 0x7fd7f41045b0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7e00052d0 con 0x7fd7f41045b0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0003e10 con 0x7fd7f41045b0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.924+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f41a1a00 con 0x7fd7f41045b0 2024-03-11T19:19:48.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.925+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f410a6e0 con 0x7fd7f41045b0 2024-03-11T19:19:48.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.927+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd7e0005440 con 0x7fd7f41045b0 2024-03-11T19:19:48.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.928+0000 7fd7f2000640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 0x7fd7dc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:48.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.928+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd7e00efbc0 con 0x7fd7f41045b0 2024-03-11T19:19:48.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.928+0000 7fd7f9400640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 0x7fd7dc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:48.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.929+0000 7fd7f9400640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 0x7fd7dc0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005fd0 tx=0x7fd7e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:48.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:48.931+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7e00b8b90 con 0x7fd7f41045b0 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: pgmap v329: 129 pgs: 3 creating+peering, 100 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/323680272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3358406785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.126+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f410f020 con 0x7fd7f41045b0 2024-03-11T19:19:49.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.129+0000 7fd7f2000640 1 -- 172.21.15.72:0/1349821981 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7fd7e00bc250 con 0x7fd7f41045b0 2024-03-11T19:19:49.133 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:49.133 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:19:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 msgr2=0x7fd7dc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 0x7fd7dc0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005fd0 tx=0x7fd7e800b410 comp rx=0 tx=0).stop 2024-03-11T19:19:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 msgr2=0x7fd7f4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006010 tx=0x7fd7e00040c0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 shutdown_connections 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd7dc0a1a30 0x7fd7dc0a3ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7f4104f70 0x7fd7f41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f41045b0 0x7fd7f4199990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 --2- 172.21.15.72:0/1349821981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7f41033c0 0x7fd7f4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 >> 172.21.15.72:0/1349821981 conn(0x7fd7f40fe970 msgr2=0x7fd7f41078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 shutdown_connections 2024-03-11T19:19:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.132+0000 7fd7fbe00640 1 -- 172.21.15.72:0/1349821981 wait complete. 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: pgmap v329: 129 pgs: 3 creating+peering, 100 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/323680272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.152 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3358406785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.171 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/974574318' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: pgmap v329: 129 pgs: 3 creating+peering, 100 active+clean, 26 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/323680272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3358406785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 -- 172.21.15.72:0/3064725600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 msgr2=0x7f1a74101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 --2- 172.21.15.72:0/3064725600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74101410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009900 tx=0x7f1a6402bc50 comp rx=0 tx=0).stop 2024-03-11T19:19:49.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 -- 172.21.15.72:0/3064725600 shutdown_connections 2024-03-11T19:19:49.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 --2- 172.21.15.72:0/3064725600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 0x7f1a74109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 --2- 172.21.15.72:0/3064725600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 0x7f1a74102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 --2- 172.21.15.72:0/3064725600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74101410 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 -- 172.21.15.72:0/3064725600 >> 172.21.15.72:0/3064725600 conn(0x7f1a740fc560 msgr2=0x7f1a740fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 -- 172.21.15.72:0/3064725600 shutdown_connections 2024-03-11T19:19:49.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.378+0000 7f1a78c00640 1 -- 172.21.15.72:0/3064725600 wait complete. 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 Processor -- start 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 -- start start 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 0x7f1a741997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 0x7f1a741a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741a0d70 con 0x7f1a74100ff0 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741a0ee0 con 0x7f1a741021e0 2024-03-11T19:19:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.379+0000 7f1a78c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741a10a0 con 0x7f1a74102ba0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54936/0 (socket says 172.21.15.72:54936) 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 -- 172.21.15.72:0/2347815293 learned_addr learned my addr 172.21.15.72:0/2347815293 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a72a00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 0x7f1a741997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73e00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 0x7f1a741a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 -- 172.21.15.72:0/2347815293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 msgr2=0x7f1a741a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 0x7f1a741a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 -- 172.21.15.72:0/2347815293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 msgr2=0x7f1a741997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 0x7f1a741997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 -- 172.21.15.72:0/2347815293 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009560 con 0x7f1a74100ff0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.380+0000 7f1a73400640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005ab0 tx=0x7f1a64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.381+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004aa0 con 0x7f1a74100ff0 2024-03-11T19:19:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.381+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a640052d0 con 0x7f1a74100ff0 2024-03-11T19:19:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.381+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a741a1320 con 0x7f1a74100ff0 2024-03-11T19:19:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.381+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64003e80 con 0x7f1a74100ff0 2024-03-11T19:19:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.381+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a741a1880 con 0x7f1a74100ff0 2024-03-11T19:19:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.382+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7410c9c0 con 0x7f1a74100ff0 2024-03-11T19:19:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.384+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1a64005440 con 0x7f1a74100ff0 2024-03-11T19:19:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.385+0000 7f1a6be00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 0x7f1a580a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.385+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1a640efb10 con 0x7f1a74100ff0 2024-03-11T19:19:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.385+0000 7f1a72a00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 0x7f1a580a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.386+0000 7f1a72a00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 0x7f1a580a3f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a7419a790 tx=0x7f1a60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:49.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.389+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a640b8a80 con 0x7f1a74100ff0 2024-03-11T19:19:49.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.626+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a7410cba0 con 0x7f1a74100ff0 2024-03-11T19:19:49.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.627+0000 7f1a6be00640 1 -- 172.21.15.72:0/2347815293 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11286 (secure 0 0 0) 0x7f1a640bc140 con 0x7f1a74100ff0 2024-03-11T19:19:49.632 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:49.632 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:19:47.263762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T19:19:49.633 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 17 2024-03-11T19:19:49.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 msgr2=0x7f1a580a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 0x7f1a580a3f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a7419a790 tx=0x7f1a60005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:49.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 msgr2=0x7f1a74199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005ab0 tx=0x7f1a64004270 comp rx=0 tx=0).stop 2024-03-11T19:19:49.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 shutdown_connections 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a580a1ab0 0x7f1a580a3f70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a74102ba0 0x7f1a741a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a741021e0 0x7f1a741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 --2- 172.21.15.72:0/2347815293 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a74100ff0 0x7f1a74199280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 >> 172.21.15.72:0/2347815293 conn(0x7f1a740fc560 msgr2=0x7f1a740733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 shutdown_connections 2024-03-11T19:19:49.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.630+0000 7f1a78c00640 1 -- 172.21.15.72:0/2347815293 wait complete. 2024-03-11T19:19:49.654 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:19:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 -- 172.21.15.72:0/2760794997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 msgr2=0x7fbd5c111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 --2- 172.21.15.72:0/2760794997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c111df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009ca0 tx=0x7fbd5402c100 comp rx=0 tx=0).stop 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 -- 172.21.15.72:0/2760794997 shutdown_connections 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 --2- 172.21.15.72:0/2760794997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c111df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 --2- 172.21.15.72:0/2760794997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 --2- 172.21.15.72:0/2760794997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 -- 172.21.15.72:0/2760794997 >> 172.21.15.72:0/2760794997 conn(0x7fbd5c06e390 msgr2=0x7fbd5c0707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 -- 172.21.15.72:0/2760794997 shutdown_connections 2024-03-11T19:19:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.862+0000 7fbd63400640 1 -- 172.21.15.72:0/2760794997 wait complete. 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 Processor -- start 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 -- start start 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c10edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c10b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c10c0c0 con 0x7fbd5c10fa00 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c10c230 con 0x7fbd5c108180 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.863+0000 7fbd63400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c10c3f0 con 0x7fbd5c108ae0 2024-03-11T19:19:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd61400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c10b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd60a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c10edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd60a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c10edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:39972/0 (socket says 172.21.15.72:39972) 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd60a00640 1 -- 172.21.15.72:0/4200058889 learned_addr learned my addr 172.21.15.72:0/4200058889 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 -- 172.21.15.72:0/4200058889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 msgr2=0x7fbd5c10edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c10edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 -- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 msgr2=0x7fbd5c10b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c10b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 -- 172.21.15.72:0/4200058889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54009900 con 0x7fbd5c108ae0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.864+0000 7fbd53e00640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9f0 tx=0x7fbd4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.865+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400ccc0 con 0x7fbd5c108ae0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.865+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c1ae150 con 0x7fbd5c108ae0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.865+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd4400ce20 con 0x7fbd5c108ae0 2024-03-11T19:19:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.865+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44004a00 con 0x7fbd5c108ae0 2024-03-11T19:19:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.865+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1ae660 con 0x7fbd5c108ae0 2024-03-11T19:19:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.866+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c1153a0 con 0x7fbd5c108ae0 2024-03-11T19:19:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.868+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbd440050b0 con 0x7fbd5c108ae0 2024-03-11T19:19:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.869+0000 7fbd5a000640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 0x7fbd400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.869+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbd44014070 con 0x7fbd5c108ae0 2024-03-11T19:19:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.869+0000 7fbd60a00640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 0x7fbd400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.870+0000 7fbd60a00640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 0x7fbd400a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:49.873+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd44096520 con 0x7fbd5c108ae0 2024-03-11T19:19:49.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:49 smithi072 ceph-mon[31695]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:19:49.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:49 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:19:49.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1349821981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:49.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2347815293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.071+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd5c1aeb10 con 0x7fbd5c108ae0 2024-03-11T19:19:50.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.073+0000 7fbd5a000640 1 -- 172.21.15.72:0/4200058889 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7fbd44099be0 con 0x7fbd5c108ae0 2024-03-11T19:19:50.077 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:50.077 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 msgr2=0x7fbd400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 0x7fbd400a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c00b4a0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 msgr2=0x7fbd5c10f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9f0 tx=0x7fbd4400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 shutdown_connections 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbd400a1a30 0x7fbd400a3ef0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd5c10fa00 0x7fbd5c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5c108ae0 0x7fbd5c10f300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 --2- 172.21.15.72:0/4200058889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c108180 0x7fbd5c10edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 >> 172.21.15.72:0/4200058889 conn(0x7fbd5c06e390 msgr2=0x7fbd5c06e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 shutdown_connections 2024-03-11T19:19:50.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.076+0000 7fbd63400640 1 -- 172.21.15.72:0/4200058889 wait complete. 2024-03-11T19:19:50.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-11T19:19:41.837094+0000', 'modified': '2024-03-11T19:19:47.263762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/806155003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 806155003}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 806155003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-11T19:19:50.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-11T19:19:50.101 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:19:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:49 smithi106 ceph-mon[35889]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:19:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:49 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:19:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1349821981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:50.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2347815293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:50.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:49 smithi143 ceph-mon[36013]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:19:50.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:49 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:19:50.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1349821981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:50.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2347815293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:19:50.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 -- 172.21.15.72:0/1454350354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28104410 msgr2=0x7efc28106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 --2- 172.21.15.72:0/1454350354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28104410 0x7efc28106800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc14009900 tx=0x7efc1402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 -- 172.21.15.72:0/1454350354 shutdown_connections 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 --2- 172.21.15.72:0/1454350354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc28109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 --2- 172.21.15.72:0/1454350354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28104410 0x7efc28106800 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 --2- 172.21.15.72:0/1454350354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28101ae0 0x7efc28103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 -- 172.21.15.72:0/1454350354 >> 172.21.15.72:0/1454350354 conn(0x7efc280fb4f0 msgr2=0x7efc280fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 -- 172.21.15.72:0/1454350354 shutdown_connections 2024-03-11T19:19:50.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.310+0000 7efc2ec00640 1 -- 172.21.15.72:0/1454350354 wait complete. 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.311+0000 7efc2ec00640 1 Processor -- start 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.311+0000 7efc2ec00640 1 -- start start 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28104410 0x7efc28197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc2819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819ec00 con 0x7efc28106d40 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819ed70 con 0x7efc28101ae0 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819ef30 con 0x7efc28104410 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40002/0 (socket says 172.21.15.72:40002) 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 -- 172.21.15.72:0/515731346 learned_addr learned my addr 172.21.15.72:0/515731346 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2ce00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28104410 0x7efc28197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2e200640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc2819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 -- 172.21.15.72:0/515731346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28104410 msgr2=0x7efc28197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28104410 0x7efc28197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 -- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 msgr2=0x7efc2819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc2819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2d800640 1 -- 172.21.15.72:0/515731346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc14009560 con 0x7efc28101ae0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.312+0000 7efc2e200640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc2819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc2d800640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9f0 tx=0x7efc1800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc26000640 1 -- 172.21.15.72:0/515731346 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18009720 con 0x7efc28101ae0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2819f210 con 0x7efc28101ae0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2819f6f0 con 0x7efc28101ae0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc26000640 1 -- 172.21.15.72:0/515731346 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc1800cd80 con 0x7efc28101ae0 2024-03-11T19:19:50.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.313+0000 7efc26000640 1 -- 172.21.15.72:0/515731346 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18010430 con 0x7efc28101ae0 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.317+0000 7efc32e5a740 1 Processor -- start 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.317+0000 7efc32e5a740 1 -- start start 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc26000640 1 -- 172.21.15.72:0/515731346 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efc180046c0 con 0x7efc28101ae0 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 0x55996d9f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 0x55996d9f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d960b30 con 0x55996d9f1760 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d95e060 con 0x55996d9f2150 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc32e5a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d959840 con 0x55996d9f72c0 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 0x55996d9f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 0x55996d9f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 0x55996d9f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54978/0 (socket says 172.21.15.72:54978) 2024-03-11T19:19:50.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2e200640 1 -- 172.21.15.72:0/1306077715 learned_addr learned my addr 172.21.15.72:0/1306077715 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:50.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 -- 172.21.15.72:0/1306077715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 msgr2=0x55996d9f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 -- 172.21.15.72:0/1306077715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 msgr2=0x55996d9f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 0x55996d9f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.318+0000 7efc2ce00640 1 -- 172.21.15.72:0/1306077715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996d9214c0 con 0x55996d9f72c0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.319+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 0x55996d9f9750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc14036040 tx=0x7efc14005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05e5bdbc44ae3b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.319+0000 7efc1ea00640 1 -- 172.21.15.72:0/1306077715 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1402ca20 con 0x55996d9f72c0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.319+0000 7efc1ea00640 1 -- 172.21.15.72:0/1306077715 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc1402cb80 con 0x55996d9f72c0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.319+0000 7efc1ea00640 1 -- 172.21.15.72:0/1306077715 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc140051d0 con 0x55996d9f72c0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.319+0000 7efc26000640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 0x7efc0c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc26000640 1 -- 172.21.15.72:0/515731346 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efc18014070 con 0x7efc28101ae0 2024-03-11T19:19:50.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc2e200640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 0x7efc0c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 -- 172.21.15.72:0/1306077715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 msgr2=0x55996d9f9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 0x55996d9f9750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc14036040 tx=0x7efc14005d50 comp rx=0 tx=0).stop 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 -- 172.21.15.72:0/1306077715 shutdown_connections 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f72c0 0x55996d9f9750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1306077715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f1760 0x55996d9f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 -- 172.21.15.72:0/1306077715 >> 172.21.15.72:0/1306077715 conn(0x55996d9eccd0 msgr2=0x55996d9ed0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 -- 172.21.15.72:0/1306077715 shutdown_connections 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc2e200640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 0x7efc0c0a3ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55996d9f29b0 tx=0x7efc20009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.320+0000 7efc32e5a740 1 -- 172.21.15.72:0/1306077715 wait complete. 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 -- start start 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 Processor -- start 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d8feea0 con 0x55996d9f72c0 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d8feb40 con 0x55996d9f2150 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d6a1f20 con 0x55996d9f1760 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.321+0000 7efc32e5a740 10 client.0 _ll_register_callbacks cb 0x55996d912950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54994/0 (socket says 172.21.15.72:54994) 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 -- 172.21.15.72:0/2931509593 learned_addr learned my addr 172.21.15.72:0/2931509593 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2ce00640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 -- 172.21.15.72:0/2931509593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 msgr2=0x55996d9fb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 -- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 msgr2=0x55996d9fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 -- 172.21.15.72:0/2931509593 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc28109d70 con 0x55996d9f72c0 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc2e200640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc20012010 tx=0x7efc20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e997e58fc0648bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.322+0000 7efc13400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20016070 con 0x55996d9f72c0 2024-03-11T19:19:50.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efc2001fc80 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55996d97dc40 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc2001fe40 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc32e5a740 10 client.14577 fetch_fsmap learned FSMap version 17 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55996d957ec0 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55996d97dc40 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc20027410 con 0x55996d9f72c0 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.323+0000 7efc32e5a740 10 client.14577 fetch_fsmap finished waiting for FSMap version 17 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc32e5a740 10 client.14577 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc32e5a740 20 client.14577 populate_metadata read hostname 'smithi072' 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da07b90 0x55996da06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc32e5a740 4 client.14577 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55996d6596c0 con 0x55996da07b90 2024-03-11T19:19:50.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc2d800640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da07b90 0x55996da06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc2d800640 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da07b90 0x55996da06db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fb59cfa1a0d636 server_cookie=cb7c3b7ce45d8a59 in_seq=0 out_seq=0 2024-03-11T19:19:50.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.324+0000 7efc13400640 10 client.14577 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.325+0000 7efc13400640 1 -- 172.21.15.72:0/2931509593 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55996d6596c0 con 0x55996da07b90 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.325+0000 7efc13400640 10 client.14577 handle_command_reply: tid=0 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.325+0000 7efc32e5a740 1 client.14577 shutdown 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.325+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 msgr2=0x55996da02a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.325+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc20012010 tx=0x7efc20004ab0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 shutdown_connections 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da07b90 0x55996da06db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996da02a70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f2150 0x55996d9fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 --2- 172.21.15.72:0/2931509593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc13400640 0 client.14577 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 >> 172.21.15.72:0/2931509593 conn(0x55996d9eccd0 msgr2=0x55996d9ebe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 shutdown_connections 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 1 -- 172.21.15.72:0/2931509593 wait complete. 2024-03-11T19:19:50.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.326+0000 7efc32e5a740 20 client.14577 trim_cache size 0 max 16384 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 Processor -- start 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 -- start start 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f2150 0x55996da02990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996d9eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996da05960 con 0x55996d9f72c0 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d6596c0 con 0x55996d9f1760 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc32e5a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996d97dc40 con 0x55996d9f2150 2024-03-11T19:19:50.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc2d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc2d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40020/0 (socket says 172.21.15.72:40020) 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc2d800640 1 -- 172.21.15.72:0/483048921 learned_addr learned my addr 172.21.15.72:0/483048921 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc2e200640 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996d9eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.343+0000 7efc2ce00640 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f2150 0x55996da02990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 -- 172.21.15.72:0/483048921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f2150 msgr2=0x55996da02990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f2150 0x55996da02990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 -- 172.21.15.72:0/483048921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 msgr2=0x55996d9eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996d9eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 -- 172.21.15.72:0/483048921 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996d8f48b0 con 0x55996d9f1760 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc2d800640 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc180971b0 tx=0x7efc18085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9638688c4b805c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc1e000640 1 -- 172.21.15.72:0/483048921 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180d0ea0 con 0x55996d9f1760 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc1e000640 1 -- 172.21.15.72:0/483048921 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc1809a320 con 0x55996d9f1760 2024-03-11T19:19:50.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.344+0000 7efc1e000640 1 -- 172.21.15.72:0/483048921 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180a3010 con 0x55996d9f1760 2024-03-11T19:19:50.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- 172.21.15.72:0/483048921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 msgr2=0x55996d9ed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc180971b0 tx=0x7efc18085040 comp rx=0 tx=0).stop 2024-03-11T19:19:50.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- 172.21.15.72:0/483048921 shutdown_connections 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f72c0 0x55996d9eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f2150 0x55996da02990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 --2- 172.21.15.72:0/483048921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f1760 0x55996d9ed0b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- 172.21.15.72:0/483048921 >> 172.21.15.72:0/483048921 conn(0x55996d9eccd0 msgr2=0x55996d9fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- 172.21.15.72:0/483048921 shutdown_connections 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- 172.21.15.72:0/483048921 wait complete. 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 -- start start 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.345+0000 7efc32e5a740 1 Processor -- start 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f2150 0x55996da00450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f72c0 0x55996da01460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996da019a0 con 0x55996d9f2150 2024-03-11T19:19:50.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996da01b10 con 0x55996d9f72c0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996da01cd0 con 0x55996d9f1760 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc32e5a740 10 client.0 _ll_register_callbacks cb 0x7efc20010ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56928/0 (socket says 172.21.15.72:56928) 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 -- 172.21.15.72:0/1691616363 learned_addr learned my addr 172.21.15.72:0/1691616363 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f2150 0x55996da00450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2e200640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f72c0 0x55996da01460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 -- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f72c0 msgr2=0x55996da01460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f72c0 0x55996da01460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 -- 172.21.15.72:0/1691616363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f2150 msgr2=0x55996da00450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f2150 0x55996da00450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.346+0000 7efc2d800640 1 -- 172.21.15.72:0/1691616363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2819f6f0 con 0x55996d9f1760 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc2d800640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9c0 tx=0x7efc18005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f723c259bcce0ba8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180a2ff0 con 0x55996d9f1760 2024-03-11T19:19:50.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efc1809a7e0 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55996da08bc0 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc1809aee0 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc32e5a740 10 client.24688 fetch_fsmap learned FSMap version 17 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55996d91a510 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55996da08bc0 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.347+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc180d5040 con 0x55996d9f1760 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 10 client.24688 fetch_fsmap finished waiting for FSMap version 17 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 10 client.24688 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 20 client.24688 populate_metadata read hostname 'smithi072' 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da6ba60 0x55996da67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 4 client.24688 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55996da08560 con 0x55996da6ba60 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da6ba60 0x55996da67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc2ce00640 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da6ba60 0x55996da67fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb4c279e9c87db1 server_cookie=d7f8f432d2f01dc4 in_seq=0 out_seq=0 2024-03-11T19:19:50.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.348+0000 7efc13400640 10 client.24688 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc13400640 1 -- 172.21.15.72:0/1691616363 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55996da08560 con 0x55996da6ba60 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc13400640 10 client.24688 handle_command_reply: tid=0 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 client.24688 shutdown 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 msgr2=0x55996d9fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9c0 tx=0x7efc18005c70 comp rx=0 tx=0).stop 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 shutdown_connections 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55996da6ba60 0x55996da67fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55996d9f72c0 0x55996da01460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55996d9f2150 0x55996da00450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 --2- 172.21.15.72:0/1691616363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55996d9f1760 0x55996d9fff10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 >> 172.21.15.72:0/1691616363 conn(0x55996d9eccd0 msgr2=0x55996d9eb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc13400640 0 client.24688 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 shutdown_connections 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 1 -- 172.21.15.72:0/1691616363 wait complete. 2024-03-11T19:19:50.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.349+0000 7efc32e5a740 20 client.24688 trim_cache size 0 max 16384 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24572, 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 7.5974854389999997, 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 17, 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 90, 2024-03-11T19:19:50.358 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 89, 2024-03-11T19:19:50.359 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 14.088207517000001 2024-03-11T19:19:50.359 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:19:50.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 msgr2=0x7efc0c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 0x7efc0c0a3ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55996d9f29b0 tx=0x7efc20009500 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 msgr2=0x7efc28197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9f0 tx=0x7efc1800edf0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 shutdown_connections 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc0c0a1a30 0x7efc0c0a3ef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc28106d40 0x7efc2819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc28104410 0x7efc28197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 --2- 172.21.15.72:0/515731346 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc28101ae0 0x7efc28197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 >> 172.21.15.72:0/515731346 conn(0x7efc280fb4f0 msgr2=0x7efc28102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 shutdown_connections 2024-03-11T19:19:50.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:50.350+0000 7efc2ec00640 1 -- 172.21.15.72:0/515731346 wait complete. 2024-03-11T19:19:50.371 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T19:19:50.371 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:50.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-11T19:19:50.774 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:19:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:50 smithi106 ceph-mon[35889]: pgmap v330: 129 pgs: 3 creating+peering, 121 active+clean, 5 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 774 B/s wr, 2 op/s 2024-03-11T19:19:51.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:50 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/4200058889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:51.155 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:50 smithi072 ceph-mon[31695]: pgmap v330: 129 pgs: 3 creating+peering, 121 active+clean, 5 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 774 B/s wr, 2 op/s 2024-03-11T19:19:51.155 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:50 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4200058889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:50 smithi143 ceph-mon[36013]: pgmap v330: 129 pgs: 3 creating+peering, 121 active+clean, 5 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 774 B/s wr, 2 op/s 2024-03-11T19:19:51.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:50 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/4200058889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:53.333 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:53 smithi072 ceph-mon[31695]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 1 op/s 2024-03-11T19:19:53.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 -- 172.21.15.72:0/1596574763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b806a2c0 msgr2=0x7fe8b810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:53.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 --2- 172.21.15.72:0/1596574763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b806a2c0 0x7fe8b810eb00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b0d0 tx=0x7fe8b000b3e0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 -- 172.21.15.72:0/1596574763 shutdown_connections 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 --2- 172.21.15.72:0/1596574763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b806a2c0 0x7fe8b810eb00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 --2- 172.21.15.72:0/1596574763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 0x7fe8b81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 --2- 172.21.15.72:0/1596574763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b8069920 0x7fe8b8069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 -- 172.21.15.72:0/1596574763 >> 172.21.15.72:0/1596574763 conn(0x7fe8b8077aa0 msgr2=0x7fe8b8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 -- 172.21.15.72:0/1596574763 shutdown_connections 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.369+0000 7fe8c0000700 1 -- 172.21.15.72:0/1596574763 wait complete. 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.370+0000 7fe8c0000700 1 Processor -- start 2024-03-11T19:19:53.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 -- start start 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b806a2c0 0x7fe8b819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 0x7fe8b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b819f390 con 0x7fe8b8069920 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b819f500 con 0x7fe8b806a2c0 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8c0000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b819f6c0 con 0x7fe8b8107cc0 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8bd600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8bd600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60108/0 (socket says 172.21.15.72:60108) 2024-03-11T19:19:53.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8bd600700 1 -- 172.21.15.72:0/2592436034 learned_addr learned my addr 172.21.15.72:0/2592436034 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.371+0000 7fe8be000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 0x7fe8b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bcc00700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b806a2c0 0x7fe8b819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 -- 172.21.15.72:0/2592436034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 msgr2=0x7fe8b819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 0x7fe8b819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 -- 172.21.15.72:0/2592436034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b806a2c0 msgr2=0x7fe8b819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b806a2c0 0x7fe8b819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 -- 172.21.15.72:0/2592436034 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009d00 con 0x7fe8b8069920 2024-03-11T19:19:53.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8bd600700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9a0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac009790 con 0x7fe8b8069920 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8ac010430 con 0x7fe8b8069920 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac018680 con 0x7fe8b8069920 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.372+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b819f9a0 con 0x7fe8b8069920 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.373+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81a7200 con 0x7fe8b8069920 2024-03-11T19:19:53.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.374+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b8068930 con 0x7fe8b8069920 2024-03-11T19:19:53.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.376+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe8ac010910 con 0x7fe8b8069920 2024-03-11T19:19:53.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.377+0000 7fe8b6000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 0x7fe8a00a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:53.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.377+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8ac014070 con 0x7fe8b8069920 2024-03-11T19:19:53.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.378+0000 7fe8bcc00700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 0x7fe8a00a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:53.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.378+0000 7fe8bcc00700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 0x7fe8a00a4000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8102b00 tx=0x7fe8a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:53.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.380+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8ac096bf0 con 0x7fe8b8069920 2024-03-11T19:19:53.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:53 smithi106 ceph-mon[35889]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 1 op/s 2024-03-11T19:19:53.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:53 smithi143 ceph-mon[36013]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 1 op/s 2024-03-11T19:19:53.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.646+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8b8111ca0 con 0x7fe8b8069920 2024-03-11T19:19:53.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.647+0000 7fe8b6000700 1 -- 172.21.15.72:0/2592436034 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe8ac026070 con 0x7fe8b8069920 2024-03-11T19:19:53.651 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T19:19:53.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 msgr2=0x7fe8a00a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:53.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 0x7fe8a00a4000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8102b00 tx=0x7fe8a4009260 comp rx=0 tx=0).stop 2024-03-11T19:19:53.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 msgr2=0x7fe8b819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:53.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9a0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 shutdown_connections 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe8a00a1b40 0x7fe8a00a4000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b8107cc0 0x7fe8b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8b8069920 0x7fe8b819a4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 --2- 172.21.15.72:0/2592436034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8b806a2c0 0x7fe8b819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 >> 172.21.15.72:0/2592436034 conn(0x7fe8b8077aa0 msgr2=0x7fe8b810ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 shutdown_connections 2024-03-11T19:19:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:53.650+0000 7fe8c0000700 1 -- 172.21.15.72:0/2592436034 wait complete. 2024-03-11T19:19:54.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:54 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2592436034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:19:54.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:54 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2592436034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:19:54.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:54 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2592436034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:54.601 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-11T19:19:54.602 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T19:19:54.602 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T19:19:54.602 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T19:19:54.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:54.602 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2024-03-11T19:19:54.657 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:19:54.658 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 41974sec preferred_lft 41974sec 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2024-03-11T19:19:54.659 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:19:54.660 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T19:19:54.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:54.660 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:19:54.660 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:19:54.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2024-03-11T19:19:54.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2024-03-11T19:19:54.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2024-03-11T19:19:54.661 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T19:19:54.661 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:19:55.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:55.232 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T19:19:55.308 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-03-11T19:19:55.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:55.312 DEBUG:teuthology.orchestra.run.smithi072:> route 2024-03-11T19:19:55.369 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2024-03-11T19:19:55.369 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T19:19:55.369 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T19:19:55.369 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T19:19:55.369 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T19:19:55.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T19:19:55.370 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:19:55.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:55 smithi106 ceph-mon[35889]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-03-11T19:19:55.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:55 smithi072 ceph-mon[31695]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-03-11T19:19:55.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:55 smithi143 ceph-mon[36013]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2024-03-11T19:19:56.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:56.320 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-03-11T19:19:56.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:56.377 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2024-03-11T19:19:56.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:56.429 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:19:56.429 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:19:56.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:19:56.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-11T19:19:56.429 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:19:56.569 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-11T19:19:56.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-11T19:19:56.570 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:19:56.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:56.895 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:19:56.895 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:19:56.895 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2024-03-11T19:19:56.895 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T19:19:56.895 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:19:57.059 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-11T19:19:57.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:19:57.060 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:57.110 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T19:19:57.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:19:57.110 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:57.159 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2024-03-11T19:19:57.244 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T19:19:57.292 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/bpf 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2024-03-11T19:19:57.293 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/abcb914a2ab7e0fc8a9eb621462ddeaa6843a6115f316745c2b0c3f1f3f3cb6e/merged 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4341691cae7cf82ab538c7711fb5534158499bac5e55f7b4b6ea344453a87a19/merged 2024-03-11T19:19:57.294 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/0 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/ed08656011a3ef0cc7d4e3e9e3030a9045200a1d8bd809ad5da79a8d8140dc18/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/b422853ad27f7f886b74a1981936f954e6ccccbc9d1093e9184a7def210a4ee1/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1d478a2d57f001a5b05e9e036d7e2b5fdeb1f91f2e70f5a7d6d7df255e730b1b/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/c9a8b852fc5ab2cddbcbf8ae961ab8f0424bf1749cc87549ed8d381c72208eac/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/03edf71624a093eeacb1cb719e6cbfb3e3c5c2656422e131c3b7ce8d417473ef/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4c23dfa9de74b82aa67a96b80b541c9518199ca3700a7eb1395f5e75c7a91224/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1435c185738bf19d370118504e38ece8322c8293169ccba79b215540524ab538/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/0e052e39c069ed70ba69911de811cbf2ac05b0311f0d8cd328d774f460ea08ec/merged 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:19:57.295 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:19:57.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:57.296 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2024-03-11T19:19:57.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-11T19:19:57.342 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-11T19:19:57.347 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2024-03-11T19:19:57.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:57 smithi106 ceph-mon[35889]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 0 op/s 2024-03-11T19:19:57.408 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:57 smithi072 ceph-mon[31695]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 0 op/s 2024-03-11T19:19:57.420 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T19:19:57.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:57 smithi143 ceph-mon[36013]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 0 op/s 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2024-03-11T19:19:57.463 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/bpf 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2024-03-11T19:19:57.464 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/abcb914a2ab7e0fc8a9eb621462ddeaa6843a6115f316745c2b0c3f1f3f3cb6e/merged 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4341691cae7cf82ab538c7711fb5534158499bac5e55f7b4b6ea344453a87a19/merged 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/0 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/ed08656011a3ef0cc7d4e3e9e3030a9045200a1d8bd809ad5da79a8d8140dc18/merged 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/b422853ad27f7f886b74a1981936f954e6ccccbc9d1093e9184a7def210a4ee1/merged 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1d478a2d57f001a5b05e9e036d7e2b5fdeb1f91f2e70f5a7d6d7df255e730b1b/merged 2024-03-11T19:19:57.465 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/c9a8b852fc5ab2cddbcbf8ae961ab8f0424bf1749cc87549ed8d381c72208eac/merged 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/03edf71624a093eeacb1cb719e6cbfb3e3c5c2656422e131c3b7ce8d417473ef/merged 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4c23dfa9de74b82aa67a96b80b541c9518199ca3700a7eb1395f5e75c7a91224/merged 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1435c185738bf19d370118504e38ece8322c8293169ccba79b215540524ab538/merged 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/0e052e39c069ed70ba69911de811cbf2ac05b0311f0d8cd328d774f460ea08ec/merged 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:19:57.466 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:19:57.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:57.467 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2024-03-11T19:19:57.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:2024-03-11T19:19:57.538+0000 7f88046b3480 -1 init, newargv = 0x5562ea62ee60 newargc=15 2024-03-11T19:19:57.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[75636]: starting ceph client 2024-03-11T19:19:57.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[75636]: starting fuse 2024-03-11T19:19:57.555 INFO:teuthology.orchestra.run.smithi072.stdout:118 2024-03-11T19:19:57.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> sudo stdin-killer -- python3 -c ' 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> import glob 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> import re 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> import os 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> import subprocess 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> def _find_admin_socket(client_name): 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> files = glob.glob(asok_path) 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> 2024-03-11T19:19:57.556 DEBUG:teuthology.orchestra.run.smithi072:> # Given a non-glob path, it better be there 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> if "*" not in asok_path: 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> assert(len(files) == 1) 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> return files[0] 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> for f in files: 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> contents = proc_f.read() 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> if mountpoint in contents: 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> return f 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> print(_find_admin_socket("client.admin")) 2024-03-11T19:19:57.557 DEBUG:teuthology.orchestra.run.smithi072:> ' 2024-03-11T19:19:57.749 INFO:teuthology.orchestra.run.smithi072.stdout:/var/run/ceph/ceph-client.admin.75636.asok 2024-03-11T19:19:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-11T19:19:57.765 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75636.asok 2024-03-11T19:19:57.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:57.765 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75636.asok status 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "metadata": { 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_sha1": "94a539707733325322964c6a187c6a1b41be1daf", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "entity_id": "admin", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": "smithi072", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "pid": "75636", 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "root": "/" 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:19:57.983 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_count": 0, 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_pinned_count": 0, 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24700, 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "inst": { 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": { 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "client", 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "num": 24700 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3109672470 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2024-03-11T19:19:57.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3109672470 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "inst_str": "client.24700 172.21.15.72:0/3109672470", 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "addr_str": "172.21.15.72:0/3109672470", 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "inode_count": 1, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_epoch": 17, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch": 90, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch_barrier": 0, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "blocklisted": false, 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs" 2024-03-11T19:19:57.985 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:19:57.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:57.996 DEBUG:teuthology.orchestra.run.smithi072:> stat --file-system '--printf=%T 2024-03-11T19:19:57.996 DEBUG:teuthology.orchestra.run.smithi072:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:58.044 INFO:teuthology.orchestra.run.smithi072.stdout:fuseblk 2024-03-11T19:19:58.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:58.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:58.044 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:19:58.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:19:58.125 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-11T19:19:58.206 INFO:teuthology.orchestra.run.smithi072.stdout:# file: . 2024-03-11T19:19:58.206 INFO:teuthology.orchestra.run.smithi072.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-11T19:19:58.206 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:58.207 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:19:58.207 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:19:58.253 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:19:58.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.495+0000 7f3397a00640 1 -- 172.21.15.72:0/3357607988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390075b40 msgr2=0x7f3390075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.495+0000 7f3397a00640 1 --2- 172.21.15.72:0/3357607988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390075b40 0x7f3390075f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3384009860 tx=0x7f338402bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:58.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 -- 172.21.15.72:0/3357607988 shutdown_connections 2024-03-11T19:19:58.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 --2- 172.21.15.72:0/3357607988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390077720 0x7f3390077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 --2- 172.21.15.72:0/3357607988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390076d80 0x7f33900771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 --2- 172.21.15.72:0/3357607988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390075b40 0x7f3390075f60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 -- 172.21.15.72:0/3357607988 >> 172.21.15.72:0/3357607988 conn(0x7f33900fe970 msgr2=0x7f3390100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:58.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 -- 172.21.15.72:0/3357607988 shutdown_connections 2024-03-11T19:19:58.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.496+0000 7f3397a00640 1 -- 172.21.15.72:0/3357607988 wait complete. 2024-03-11T19:19:58.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 Processor -- start 2024-03-11T19:19:58.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 -- start start 2024-03-11T19:19:58.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390075b40 0x7f339019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 0x7f33901a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901a5310 con 0x7f3390075b40 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901a5480 con 0x7f3390077720 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.497+0000 7f3397a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901a5640 con 0x7f3390076d80 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:60832/0 (socket says 172.21.15.72:60832) 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 -- 172.21.15.72:0/3181475790 learned_addr learned my addr 172.21.15.72:0/3181475790 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f3395000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390075b40 0x7f339019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f3395a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 0x7f33901a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 -- 172.21.15.72:0/3181475790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 msgr2=0x7f33901a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 0x7f33901a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 -- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390075b40 msgr2=0x7f339019d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390075b40 0x7f339019d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 -- 172.21.15.72:0/3181475790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009490 con 0x7f3390076d80 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f3395a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 0x7f33901a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.498+0000 7f338fe00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e9c0 tx=0x7f337c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.499+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00ccc0 con 0x7f3390076d80 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.499+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33901a5920 con 0x7f3390076d80 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.499+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f337c00ce20 con 0x7f3390076d80 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.499+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c010430 con 0x7f3390076d80 2024-03-11T19:19:58.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.499+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33901a5e80 con 0x7f3390076d80 2024-03-11T19:19:58.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.500+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339010fee0 con 0x7f3390076d80 2024-03-11T19:19:58.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.505+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f337c004f80 con 0x7f3390076d80 2024-03-11T19:19:58.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.506+0000 7f338d600640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 0x7f33780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.506+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f337c014070 con 0x7f3390076d80 2024-03-11T19:19:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.506+0000 7f3395000640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 0x7f33780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.506+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f337c0fadc0 con 0x7f3390076d80 2024-03-11T19:19:58.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.507+0000 7f3395000640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 0x7f33780a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f338402c180 tx=0x7f3384034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:58.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.703+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3390113180 con 0x7f3390076d80 2024-03-11T19:19:58.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.705+0000 7f338d600640 1 -- 172.21.15.72:0/3181475790 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7f337c096af0 con 0x7f3390076d80 2024-03-11T19:19:58.710 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:19:58.710 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 msgr2=0x7f33780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 0x7f33780a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f338402c180 tx=0x7f3384034c00 comp rx=0 tx=0).stop 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 msgr2=0x7f339019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f337c00e9c0 tx=0x7f337c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 shutdown_connections 2024-03-11T19:19:58.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33780a1a30 0x7f33780a3ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3390077720 0x7f33901a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3390076d80 0x7f339019dd60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 --2- 172.21.15.72:0/3181475790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3390075b40 0x7f339019d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 >> 172.21.15.72:0/3181475790 conn(0x7f33900fe970 msgr2=0x7f3390100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 shutdown_connections 2024-03-11T19:19:58.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.708+0000 7f3397a00640 1 -- 172.21.15.72:0/3181475790 wait complete. 2024-03-11T19:19:58.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:19:58.730 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 -- 172.21.15.72:0/656162330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16901076a0 msgr2=0x7f1690107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 --2- 172.21.15.72:0/656162330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16901076a0 0x7f1690107ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1680009860 tx=0x7f168002bb70 comp rx=0 tx=0).stop 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 -- 172.21.15.72:0/656162330 shutdown_connections 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 --2- 172.21.15.72:0/656162330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f169010fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 --2- 172.21.15.72:0/656162330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1690108890 0x7f1690108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 --2- 172.21.15.72:0/656162330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16901076a0 0x7f1690107ac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 -- 172.21.15.72:0/656162330 >> 172.21.15.72:0/656162330 conn(0x7f1690076650 msgr2=0x7f1690078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 -- 172.21.15.72:0/656162330 shutdown_connections 2024-03-11T19:19:58.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.923+0000 7f168fe00640 1 -- 172.21.15.72:0/656162330 wait complete. 2024-03-11T19:19:58.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.924+0000 7f168fe00640 1 Processor -- start 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.924+0000 7f168fe00640 1 -- start start 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 0x7f169019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f16901a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a51e0 con 0x7f16901076a0 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a5350 con 0x7f1690109250 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901a5510 con 0x7f1690108890 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60142/0 (socket says 172.21.15.72:60142) 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 -- 172.21.15.72:0/415364592 learned_addr learned my addr 172.21.15.72:0/415364592 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f1686000640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 0x7f169019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168f400640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f16901a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 -- 172.21.15.72:0/415364592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 msgr2=0x7f169019dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 0x7f169019dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 -- 172.21.15.72:0/415364592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 msgr2=0x7f16901a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f16901a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168ea00640 1 -- 172.21.15.72:0/415364592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678009560 con 0x7f16901076a0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f168f400640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f16901a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.925+0000 7f1686000640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 0x7f169019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f168ea00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f168002c0b0 tx=0x7f1680005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168002c510 con 0x7f16901076a0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168002cb50 con 0x7f16901076a0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680009490 con 0x7f16901076a0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680003850 con 0x7f16901076a0 2024-03-11T19:19:58.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.926+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16901a5970 con 0x7f16901076a0 2024-03-11T19:19:58.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.928+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169010e9c0 con 0x7f16901076a0 2024-03-11T19:19:58.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.929+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f168003b040 con 0x7f16901076a0 2024-03-11T19:19:58.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.931+0000 7f1687e00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 0x7f16740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:58.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.931+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f16800ef3a0 con 0x7f16901076a0 2024-03-11T19:19:58.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.931+0000 7f1686000640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 0x7f16740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:58.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.932+0000 7f1686000640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 0x7f16740a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f169019ec00 tx=0x7f1678009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:58.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:58.935+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16800b8310 con 0x7f16901076a0 2024-03-11T19:19:59.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.129+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16901130b0 con 0x7f16901076a0 2024-03-11T19:19:59.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.130+0000 7f1687e00640 1 -- 172.21.15.72:0/415364592 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16901130b0 con 0x7f16901076a0 2024-03-11T19:19:59.135 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:19:59.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 msgr2=0x7f16740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 0x7f16740a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f169019ec00 tx=0x7f1678009260 comp rx=0 tx=0).stop 2024-03-11T19:19:59.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 msgr2=0x7f169019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f168002c0b0 tx=0x7f1680005d50 comp rx=0 tx=0).stop 2024-03-11T19:19:59.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 shutdown_connections 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f16740a1a30 0x7f16740a3ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1690109250 0x7f16901a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1690108890 0x7f169019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 --2- 172.21.15.72:0/415364592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16901076a0 0x7f169019d6f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 >> 172.21.15.72:0/415364592 conn(0x7f1690076650 msgr2=0x7f169010bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 shutdown_connections 2024-03-11T19:19:59.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.133+0000 7f168fe00640 1 -- 172.21.15.72:0/415364592 wait complete. 2024-03-11T19:19:59.156 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.360+0000 7fdf41c00640 1 -- 172.21.15.72:0/1218504537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c109350 msgr2=0x7fdf3c10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.360+0000 7fdf41c00640 1 --2- 172.21.15.72:0/1218504537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c109350 0x7fdf3c10fbe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009900 tx=0x7fdf3402bc30 comp rx=0 tx=0).stop 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 -- 172.21.15.72:0/1218504537 shutdown_connections 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 --2- 172.21.15.72:0/1218504537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c109350 0x7fdf3c10fbe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 --2- 172.21.15.72:0/1218504537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 --2- 172.21.15.72:0/1218504537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 -- 172.21.15.72:0/1218504537 >> 172.21.15.72:0/1218504537 conn(0x7fdf3c076740 msgr2=0x7fdf3c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 -- 172.21.15.72:0/1218504537 shutdown_connections 2024-03-11T19:19:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.361+0000 7fdf41c00640 1 -- 172.21.15.72:0/1218504537 wait complete. 2024-03-11T19:19:59.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 Processor -- start 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 -- start start 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c109350 0x7fdf3c1a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a52b0 con 0x7fdf3c108990 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a5420 con 0x7fdf3c109350 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf41c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c1a55e0 con 0x7fdf3c1077a0 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.362+0000 7fdf32000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60160/0 (socket says 172.21.15.72:60160) 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 -- 172.21.15.72:0/2167195022 learned_addr learned my addr 172.21.15.72:0/2167195022 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf3b400640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c109350 0x7fdf3c1a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf3aa00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 -- 172.21.15.72:0/2167195022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 msgr2=0x7fdf3c19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 -- 172.21.15.72:0/2167195022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c109350 msgr2=0x7fdf3c1a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c109350 0x7fdf3c1a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 -- 172.21.15.72:0/2167195022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009560 con 0x7fdf3c108990 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf3aa00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.363+0000 7fdf32000640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ebc0 tx=0x7fdf2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.364+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240099b0 con 0x7fdf3c108990 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.364+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c1a58c0 con 0x7fdf3c108990 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.364+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf2400eec0 con 0x7fdf3c108990 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.364+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24018b60 con 0x7fdf3c108990 2024-03-11T19:19:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.364+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c1a5e20 con 0x7fdf3c108990 2024-03-11T19:19:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.365+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c10ea60 con 0x7fdf3c108990 2024-03-11T19:19:59.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.368+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdf24018cc0 con 0x7fdf3c108990 2024-03-11T19:19:59.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.369+0000 7fdf33e00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 0x7fdf200a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.370+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdf24014070 con 0x7fdf3c108990 2024-03-11T19:19:59.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.370+0000 7fdf3aa00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 0x7fdf200a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.370+0000 7fdf3aa00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 0x7fdf200a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c1087f0 tx=0x7fdf2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:59.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.372+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf240971b0 con 0x7fdf3c108990 2024-03-11T19:19:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:59 smithi106 ceph-mon[35889]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:59 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3181475790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:59.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:19:59 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/415364592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:59 smithi072 ceph-mon[31695]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:59 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3181475790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:59.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:19:59 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/415364592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:59 smithi143 ceph-mon[36013]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:19:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:59 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3181475790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:19:59.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:19:59 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/415364592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:19:59.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.570+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf3c1133b0 con 0x7fdf3c108990 2024-03-11T19:19:59.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.570+0000 7fdf33e00640 1 -- 172.21.15.72:0/2167195022 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf24026020 con 0x7fdf3c108990 2024-03-11T19:19:59.574 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:19:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 msgr2=0x7fdf200a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 0x7fdf200a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c1087f0 tx=0x7fdf2c005f90 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 msgr2=0x7fdf3c19dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ebc0 tx=0x7fdf2400c6a0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 shutdown_connections 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf200a1a30 0x7fdf200a3ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3c109350 0x7fdf3c1a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf3c108990 0x7fdf3c19dd00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 --2- 172.21.15.72:0/2167195022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf3c1077a0 0x7fdf3c19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 >> 172.21.15.72:0/2167195022 conn(0x7fdf3c076740 msgr2=0x7fdf3c10bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.573+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 shutdown_connections 2024-03-11T19:19:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.574+0000 7fdf41c00640 1 -- 172.21.15.72:0/2167195022 wait complete. 2024-03-11T19:19:59.597 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 -- 172.21.15.72:0/1070928871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 msgr2=0x7f780c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 --2- 172.21.15.72:0/1070928871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c108e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802bc50 comp rx=0 tx=0).stop 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 -- 172.21.15.72:0/1070928871 shutdown_connections 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 --2- 172.21.15.72:0/1070928871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 0x7f780c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 --2- 172.21.15.72:0/1070928871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c108e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 --2- 172.21.15.72:0/1070928871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 -- 172.21.15.72:0/1070928871 >> 172.21.15.72:0/1070928871 conn(0x7f780c076740 msgr2=0x7f780c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 -- 172.21.15.72:0/1070928871 shutdown_connections 2024-03-11T19:19:59.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.801+0000 7f7812400640 1 -- 172.21.15.72:0/1070928871 wait complete. 2024-03-11T19:19:59.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 Processor -- start 2024-03-11T19:19:59.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 -- start start 2024-03-11T19:19:59.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 0x7f780c1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a5350 con 0x7f780c109350 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a54c0 con 0x7f780c108990 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f7812400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a5680 con 0x7f780c1077a0 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f780b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f780be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 0x7f780c1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f780b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:60862/0 (socket says 172.21.15.72:60862) 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f780b400640 1 -- 172.21.15.72:0/185298309 learned_addr learned my addr 172.21.15.72:0/185298309 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.803+0000 7f780aa00640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 -- 172.21.15.72:0/185298309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 msgr2=0x7f780c19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 -- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 msgr2=0x7f780c1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 0x7f780c1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 -- 172.21.15.72:0/185298309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8009560 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f780b400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9c0 tx=0x7f77fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0096e0 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77fc00cd80 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc010430 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.804+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c1a5960 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.805+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c1a5df0 con 0x7f780c1077a0 2024-03-11T19:19:59.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.806+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c10eac0 con 0x7f780c1077a0 2024-03-11T19:19:59.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.808+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f77fc003680 con 0x7f780c1077a0 2024-03-11T19:19:59.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.809+0000 7f7803e00640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 0x7f77f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:19:59.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.809+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f77fc014070 con 0x7f780c1077a0 2024-03-11T19:19:59.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.809+0000 7f780aa00640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 0x7f77f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:19:59.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.810+0000 7f780aa00640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 0x7f77f00a3ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:19:59.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.811+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f77fc096980 con 0x7f780c1077a0 2024-03-11T19:19:59.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.992+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f780c1134a0 con 0x7f780c1077a0 2024-03-11T19:19:59.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.993+0000 7f7803e00640 1 -- 172.21.15.72:0/185298309 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77fc09a040 con 0x7f780c1077a0 2024-03-11T19:19:59.997 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 msgr2=0x7f77f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 0x7f77f00a3ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f802c800 comp rx=0 tx=0).stop 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 msgr2=0x7f780c19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9c0 tx=0x7f77fc00ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 shutdown_connections 2024-03-11T19:20:00.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77f00a1a30 0x7f77f00a3ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f780c109350 0x7f780c1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f780c108990 0x7f780c19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 --2- 172.21.15.72:0/185298309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f780c1077a0 0x7f780c19d860 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 >> 172.21.15.72:0/185298309 conn(0x7f780c076740 msgr2=0x7f780c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 shutdown_connections 2024-03-11T19:20:00.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:19:59.996+0000 7f7812400640 1 -- 172.21.15.72:0/185298309 wait complete. 2024-03-11T19:20:00.019 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 -- 172.21.15.72:0/393364416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e41077a0 msgr2=0x7fc2e4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 --2- 172.21.15.72:0/393364416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e41077a0 0x7fc2e4107bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009900 tx=0x7fc2d802bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 -- 172.21.15.72:0/393364416 shutdown_connections 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 --2- 172.21.15.72:0/393364416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e4109350 0x7fc2e410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 --2- 172.21.15.72:0/393364416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 0x7fc2e4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 --2- 172.21.15.72:0/393364416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e41077a0 0x7fc2e4107bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 -- 172.21.15.72:0/393364416 >> 172.21.15.72:0/393364416 conn(0x7fc2e4076740 msgr2=0x7fc2e4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 -- 172.21.15.72:0/393364416 shutdown_connections 2024-03-11T19:20:00.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.227+0000 7fc2eb800640 1 -- 172.21.15.72:0/393364416 wait complete. 2024-03-11T19:20:00.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.228+0000 7fc2eb800640 1 Processor -- start 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.228+0000 7fc2eb800640 1 -- start start 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 0x7fc2e419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e4109350 0x7fc2e41a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41a5370 con 0x7fc2e41077a0 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41a54e0 con 0x7fc2e4109350 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2eb800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41a56a0 con 0x7fc2e4108990 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60200/0 (socket says 172.21.15.72:60200) 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 -- 172.21.15.72:0/3598761820 learned_addr learned my addr 172.21.15.72:0/3598761820 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e3e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 0x7fc2e419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e9800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e4109350 0x7fc2e41a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 -- 172.21.15.72:0/3598761820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 msgr2=0x7fc2e419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 0x7fc2e419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 -- 172.21.15.72:0/3598761820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e4109350 msgr2=0x7fc2e41a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e4109350 0x7fc2e41a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.229+0000 7fc2e8e00640 1 -- 172.21.15.72:0/3598761820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d8009560 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2e8e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8000c00 tx=0x7fc2d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80044e0 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e41a5920 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2d80054e0 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e41a5e80 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.230+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8003e90 con 0x7fc2e41077a0 2024-03-11T19:20:00.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.231+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e410eac0 con 0x7fc2e41077a0 2024-03-11T19:20:00.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.233+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc2d80035f0 con 0x7fc2e41077a0 2024-03-11T19:20:00.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.234+0000 7fc2e1600640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 0x7fc2cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.235+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc2d80eff20 con 0x7fc2e41077a0 2024-03-11T19:20:00.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.235+0000 7fc2e3e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 0x7fc2cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.235+0000 7fc2e3e00640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 0x7fc2cc0a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2e419ed90 tx=0x7fc2d000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:00.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.238+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2d80b8e90 con 0x7fc2e41077a0 2024-03-11T19:20:00.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2167195022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/185298309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:00 smithi072 ceph-mon[31695]: overall HEALTH_OK 2024-03-11T19:20:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2167195022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/185298309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:00 smithi143 ceph-mon[36013]: overall HEALTH_OK 2024-03-11T19:20:00.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.437+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2e41a5ab0 con 0x7fc2e41077a0 2024-03-11T19:20:00.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.438+0000 7fc2e1600640 1 -- 172.21.15.72:0/3598761820 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc2e41a5ab0 con 0x7fc2e41077a0 2024-03-11T19:20:00.442 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:20:00.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 msgr2=0x7fc2cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 0x7fc2cc0a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2e419ed90 tx=0x7fc2d000b390 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 msgr2=0x7fc2e419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8000c00 tx=0x7fc2d80042f0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 shutdown_connections 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc2cc0a1a30 0x7fc2cc0a3ef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2e4109350 0x7fc2e41a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2e4108990 0x7fc2e419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 --2- 172.21.15.72:0/3598761820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2e41077a0 0x7fc2e419d880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 >> 172.21.15.72:0/3598761820 conn(0x7fc2e4076740 msgr2=0x7fc2e410bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 shutdown_connections 2024-03-11T19:20:00.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.441+0000 7fc2eb800640 1 -- 172.21.15.72:0/3598761820 wait complete. 2024-03-11T19:20:00.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:20:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2167195022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/185298309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:00 smithi106 ceph-mon[35889]: overall HEALTH_OK 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 -- 172.21.15.72:0/3892221270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541045b0 msgr2=0x7fb754104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 --2- 172.21.15.72:0/3892221270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541045b0 0x7fb754104a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009900 tx=0x7fb73c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 -- 172.21.15.72:0/3892221270 shutdown_connections 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 --2- 172.21.15.72:0/3892221270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb754104f70 0x7fb75410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 --2- 172.21.15.72:0/3892221270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541045b0 0x7fb754104a30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 --2- 172.21.15.72:0/3892221270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7541033c0 0x7fb7541037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 -- 172.21.15.72:0/3892221270 >> 172.21.15.72:0/3892221270 conn(0x7fb7540fe930 msgr2=0x7fb754100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:00.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 -- 172.21.15.72:0/3892221270 shutdown_connections 2024-03-11T19:20:00.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.669+0000 7fb759c00640 1 -- 172.21.15.72:0/3892221270 wait complete. 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.670+0000 7fb759c00640 1 Processor -- start 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.670+0000 7fb759c00640 1 -- start start 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 0x7fb754199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 0x7fb7541a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7541a0f30 con 0x7fb7541045b0 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7541a10a0 con 0x7fb754104f70 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb759c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7541a1260 con 0x7fb7541033c0 2024-03-11T19:20:00.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb752000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb752a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 0x7fb754199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb752000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60208/0 (socket says 172.21.15.72:60208) 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb752000640 1 -- 172.21.15.72:0/941954160 learned_addr learned my addr 172.21.15.72:0/941954160 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.671+0000 7fb753400640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 0x7fb7541a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 -- 172.21.15.72:0/941954160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 msgr2=0x7fb7541a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 0x7fb7541a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 -- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 msgr2=0x7fb754199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 -- 172.21.15.72:0/941954160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c009560 con 0x7fb7541033c0 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752000640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb753400640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 0x7fb7541a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:00.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb752a00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 0x7fb754199440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9c0 tx=0x7fb74400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400ccc0 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb74400ce20 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7541a1540 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744010430 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.672+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7541a1a50 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.673+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75410a730 con 0x7fb7541033c0 2024-03-11T19:20:00.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.675+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb744004000 con 0x7fb7541033c0 2024-03-11T19:20:00.681 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.677+0000 7fb74b400640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 0x7fb7340a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:00.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.678+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb744014070 con 0x7fb7541033c0 2024-03-11T19:20:00.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.678+0000 7fb752000640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 0x7fb7340a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.679+0000 7fb752000640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 0x7fb7340a3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb75419a950 tx=0x7fb73c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:00.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.681+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7440968f0 con 0x7fb7541033c0 2024-03-11T19:20:00.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.879+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb75410edd0 con 0x7fb7541033c0 2024-03-11T19:20:00.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.880+0000 7fb74b400640 1 -- 172.21.15.72:0/941954160 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+591 (secure 0 0 0) 0x7fb744099fb0 con 0x7fb7541033c0 2024-03-11T19:20:00.884 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1618271736 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2331163134 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3661300568 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/618300141 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2881891200 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3531203560 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:00.885 INFO:teuthology.orchestra.run.smithi072.stderr:listed 10 entries 2024-03-11T19:20:00.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 msgr2=0x7fb7340a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 0x7fb7340a3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb75419a950 tx=0x7fb73c02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 msgr2=0x7fb754199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:00.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 0x7fb754199440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9c0 tx=0x7fb74400ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:00.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 shutdown_connections 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb7340a1ab0 0x7fb7340a3f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb754104f70 0x7fb7541a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7541045b0 0x7fb754199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 --2- 172.21.15.72:0/941954160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7541033c0 0x7fb754199440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 >> 172.21.15.72:0/941954160 conn(0x7fb7540fe930 msgr2=0x7fb754107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 shutdown_connections 2024-03-11T19:20:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:00.883+0000 7fb759c00640 1 -- 172.21.15.72:0/941954160 wait complete. 2024-03-11T19:20:00.905 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:20:00.905 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:20:00.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:20:01.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 -- 172.21.15.72:0/1912669302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c1021b0 msgr2=0x7fa75c102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 --2- 172.21.15.72:0/1912669302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c1021b0 0x7fa75c102630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa744009860 tx=0x7fa74402bb70 comp rx=0 tx=0).stop 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 -- 172.21.15.72:0/1912669302 shutdown_connections 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 --2- 172.21.15.72:0/1912669302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c102b70 0x7fa75c109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 --2- 172.21.15.72:0/1912669302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c1021b0 0x7fa75c102630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 --2- 172.21.15.72:0/1912669302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c100fc0 0x7fa75c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 -- 172.21.15.72:0/1912669302 >> 172.21.15.72:0/1912669302 conn(0x7fa75c0fc510 msgr2=0x7fa75c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 -- 172.21.15.72:0/1912669302 shutdown_connections 2024-03-11T19:20:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.191+0000 7fa761000640 1 -- 172.21.15.72:0/1912669302 wait complete. 2024-03-11T19:20:01.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.192+0000 7fa761000640 1 Processor -- start 2024-03-11T19:20:01.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.192+0000 7fa761000640 1 -- start start 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 0x7fa75c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c102b70 0x7fa75c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0dc0 con 0x7fa75c1021b0 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0f30 con 0x7fa75c100fc0 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa761000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a10f0 con 0x7fa75c102b70 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60226/0 (socket says 172.21.15.72:60226) 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 -- 172.21.15.72:0/1838631438 learned_addr learned my addr 172.21.15.72:0/1838631438 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa75b400640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 0x7fa75c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa75be00640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c102b70 0x7fa75c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 -- 172.21.15.72:0/1838631438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c102b70 msgr2=0x7fa75c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c102b70 0x7fa75c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 -- 172.21.15.72:0/1838631438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 msgr2=0x7fa75c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 0x7fa75c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa752a00640 1 -- 172.21.15.72:0/1838631438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c009900 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.193+0000 7fa75b400640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 0x7fa75c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa752a00640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa744034b10 tx=0x7fa7440045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744005020 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa744009490 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa74402c510 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744004a60 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.194+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1a1550 con 0x7fa75c1021b0 2024-03-11T19:20:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.195+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c10c9c0 con 0x7fa75c1021b0 2024-03-11T19:20:01.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.197+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa74403b040 con 0x7fa75c1021b0 2024-03-11T19:20:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.199+0000 7fa758c00640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 0x7fa7400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.199+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa7440eef60 con 0x7fa75c1021b0 2024-03-11T19:20:01.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.199+0000 7fa75b400640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 0x7fa7400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.200+0000 7fa75b400640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 0x7fa7400a3ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0098d0 tx=0x7fa74c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:01.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.203+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7440b7f80 con 0x7fa75c1021b0 2024-03-11T19:20:01.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.402+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa75c1a19c0 con 0x7fa75c1021b0 2024-03-11T19:20:01.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.406+0000 7fa758c00640 1 -- 172.21.15.72:0/1838631438 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7fa74404a090 con 0x7fa75c1021b0 2024-03-11T19:20:01.411 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:01.411 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.409+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 msgr2=0x7fa7400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.409+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 0x7fa7400a3ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0098d0 tx=0x7fa74c010040 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.409+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 msgr2=0x7fa75c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.409+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa744034b10 tx=0x7fa7440045c0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 shutdown_connections 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7400a1a30 0x7fa7400a3ef0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa75c102b70 0x7fa75c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa75c1021b0 0x7fa75c199810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 --2- 172.21.15.72:0/1838631438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c100fc0 0x7fa75c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 >> 172.21.15.72:0/1838631438 conn(0x7fa75c0fc510 msgr2=0x7fa75c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 shutdown_connections 2024-03-11T19:20:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.410+0000 7fa761000640 1 -- 172.21.15.72:0/1838631438 wait complete. 2024-03-11T19:20:01.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:01 smithi072 ceph-mon[31695]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:01.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:01 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3598761820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:01 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/941954160' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:01.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:20:01.435 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:20:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:01 smithi143 ceph-mon[36013]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:01 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3598761820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:01.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:01 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/941954160' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:01 smithi106 ceph-mon[35889]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:01 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3598761820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:01 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/941954160' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 -- 172.21.15.72:0/3989783121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc104f70 msgr2=0x7f23bc10b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 --2- 172.21.15.72:0/3989783121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc104f70 0x7f23bc10b800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009900 tx=0x7f23b402bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 -- 172.21.15.72:0/3989783121 shutdown_connections 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 --2- 172.21.15.72:0/3989783121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc104f70 0x7f23bc10b800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 --2- 172.21.15.72:0/3989783121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1045b0 0x7f23bc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 --2- 172.21.15.72:0/3989783121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc1033c0 0x7f23bc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 -- 172.21.15.72:0/3989783121 >> 172.21.15.72:0/3989783121 conn(0x7f23bc0fe930 msgr2=0x7f23bc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 -- 172.21.15.72:0/3989783121 shutdown_connections 2024-03-11T19:20:01.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.649+0000 7f23c3200640 1 -- 172.21.15.72:0/3989783121 wait complete. 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 Processor -- start 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 -- start start 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1033c0 0x7f23bc1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc104f70 0x7f23bc1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0ec0 con 0x7f23bc104f70 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a1030 con 0x7f23bc1045b0 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.650+0000 7f23c3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a11f0 con 0x7f23bc1033c0 2024-03-11T19:20:01.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23c1200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc104f70 0x7f23bc1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:39820/0 (socket says 172.21.15.72:39820) 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 -- 172.21.15.72:0/2786210589 learned_addr learned my addr 172.21.15.72:0/2786210589 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bbe00640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1033c0 0x7f23bc1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 -- 172.21.15.72:0/2786210589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1033c0 msgr2=0x7f23bc1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1033c0 0x7f23bc1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 -- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc104f70 msgr2=0x7f23bc1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc104f70 0x7f23bc1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 -- 172.21.15.72:0/2786210589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b4009560 con 0x7f23bc1045b0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.651+0000 7f23bb400640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9f0 tx=0x7f23a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.652+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4009720 con 0x7f23bc1045b0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.652+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23a400cd80 con 0x7f23bc1045b0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.652+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc1a14d0 con 0x7f23bc1045b0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.652+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4010690 con 0x7f23bc1045b0 2024-03-11T19:20:01.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.652+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc1a19e0 con 0x7f23bc1045b0 2024-03-11T19:20:01.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.653+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc10a680 con 0x7f23bc1045b0 2024-03-11T19:20:01.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.655+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f23a40046c0 con 0x7f23bc1045b0 2024-03-11T19:20:01.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.657+0000 7f23b8c00640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 0x7f23a00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:01.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.657+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f23a4014070 con 0x7f23bc1045b0 2024-03-11T19:20:01.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.657+0000 7f23bbe00640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 0x7f23a00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:01.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.658+0000 7f23bbe00640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 0x7f23a00a3f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f23bc104410 tx=0x7f23ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:01.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.658+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23a4096ae0 con 0x7f23bc1045b0 2024-03-11T19:20:01.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.853+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23bc10efb0 con 0x7f23bc1045b0 2024-03-11T19:20:01.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.854+0000 7f23b8c00640 1 -- 172.21.15.72:0/2786210589 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23bc10efb0 con 0x7f23bc1045b0 2024-03-11T19:20:01.859 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:20:01.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 msgr2=0x7f23a00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 0x7f23a00a3f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f23bc104410 tx=0x7f23ac009500 comp rx=0 tx=0).stop 2024-03-11T19:20:01.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 msgr2=0x7f23bc199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:01.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9f0 tx=0x7f23a400edf0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 shutdown_connections 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f23a00a1ab0 0x7f23a00a3f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23bc104f70 0x7f23bc1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23bc1045b0 0x7f23bc199910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 --2- 172.21.15.72:0/2786210589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23bc1033c0 0x7f23bc1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 >> 172.21.15.72:0/2786210589 conn(0x7f23bc0fe930 msgr2=0x7f23bc107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 shutdown_connections 2024-03-11T19:20:01.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:01.857+0000 7f23c3200640 1 -- 172.21.15.72:0/2786210589 wait complete. 2024-03-11T19:20:01.880 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 -- 172.21.15.72:0/4154944656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104f50 msgr2=0x7fba8010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 --2- 172.21.15.72:0/4154944656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104f50 0x7fba8010b7e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba78009ca0 tx=0x7fba7802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 -- 172.21.15.72:0/4154944656 shutdown_connections 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 --2- 172.21.15.72:0/4154944656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104f50 0x7fba8010b7e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 --2- 172.21.15.72:0/4154944656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104590 0x7fba80104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 --2- 172.21.15.72:0/4154944656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801033a0 0x7fba801037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.091+0000 7fba84e00640 1 -- 172.21.15.72:0/4154944656 >> 172.21.15.72:0/4154944656 conn(0x7fba800fe930 msgr2=0x7fba80100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.092+0000 7fba84e00640 1 -- 172.21.15.72:0/4154944656 shutdown_connections 2024-03-11T19:20:02.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.092+0000 7fba84e00640 1 -- 172.21.15.72:0/4154944656 wait complete. 2024-03-11T19:20:02.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 Processor -- start 2024-03-11T19:20:02.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 -- start start 2024-03-11T19:20:02.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104590 0x7fba80199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba80104f50 0x7fba801a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a0f10 con 0x7fba80104590 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a1080 con 0x7fba801033a0 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.093+0000 7fba84e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a1240 con 0x7fba80104f50 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba75400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104590 0x7fba80199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:58646/0 (socket says 172.21.15.72:58646) 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 -- 172.21.15.72:0/451345756 learned_addr learned my addr 172.21.15.72:0/451345756 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:02.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7e800640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba80104f50 0x7fba801a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 -- 172.21.15.72:0/451345756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba80104f50 msgr2=0x7fba801a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba80104f50 0x7fba801a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 -- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104590 msgr2=0x7fba80199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104590 0x7fba80199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 -- 172.21.15.72:0/451345756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009490 con 0x7fba801033a0 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba7de00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.094+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70009700 con 0x7fba801033a0 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.095+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba78009900 con 0x7fba801033a0 2024-03-11T19:20:02.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.095+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba7000cd80 con 0x7fba801033a0 2024-03-11T19:20:02.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.095+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70010430 con 0x7fba801033a0 2024-03-11T19:20:02.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.095+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801a18a0 con 0x7fba801033a0 2024-03-11T19:20:02.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.096+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8010a660 con 0x7fba801033a0 2024-03-11T19:20:02.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.098+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fba70003680 con 0x7fba801033a0 2024-03-11T19:20:02.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.099+0000 7fba77400640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 0x7fba640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.099+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fba70014070 con 0x7fba801033a0 2024-03-11T19:20:02.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.099+0000 7fba75400640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 0x7fba640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.100+0000 7fba75400640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 0x7fba640a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba8019a930 tx=0x7fba68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.101+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba70096b40 con 0x7fba801033a0 2024-03-11T19:20:02.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.305+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba8010edb0 con 0x7fba801033a0 2024-03-11T19:20:02.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.306+0000 7fba77400640 1 -- 172.21.15.72:0/451345756 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba7009a200 con 0x7fba801033a0 2024-03-11T19:20:02.310 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:20:02.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 msgr2=0x7fba640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 0x7fba640a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba8019a930 tx=0x7fba68005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 msgr2=0x7fba80199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 shutdown_connections 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1a30 0x7fba640a3ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba80104f50 0x7fba801a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba80104590 0x7fba80199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 --2- 172.21.15.72:0/451345756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba801033a0 0x7fba80199420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 >> 172.21.15.72:0/451345756 conn(0x7fba800fe930 msgr2=0x7fba80107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 shutdown_connections 2024-03-11T19:20:02.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.308+0000 7fba84e00640 1 -- 172.21.15.72:0/451345756 wait complete. 2024-03-11T19:20:02.332 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:20:02.413 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:02 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1838631438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:02.413 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:02 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2786210589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:20:02.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:02 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1838631438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:02.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:02 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2786210589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:20:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.542+0000 7fabc8000640 1 -- 172.21.15.72:0/1929521275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc0104f70 msgr2=0x7fabc010b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.542+0000 7fabc8000640 1 --2- 172.21.15.72:0/1929521275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc0104f70 0x7fabc010b800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009900 tx=0x7fabb402bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.542+0000 7fabc8000640 1 -- 172.21.15.72:0/1929521275 shutdown_connections 2024-03-11T19:20:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.542+0000 7fabc8000640 1 --2- 172.21.15.72:0/1929521275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc0104f70 0x7fabc010b800 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 --2- 172.21.15.72:0/1929521275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc01045b0 0x7fabc0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 --2- 172.21.15.72:0/1929521275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 -- 172.21.15.72:0/1929521275 >> 172.21.15.72:0/1929521275 conn(0x7fabc00fe930 msgr2=0x7fabc0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:02.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 -- 172.21.15.72:0/1929521275 shutdown_connections 2024-03-11T19:20:02.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 -- 172.21.15.72:0/1929521275 wait complete. 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.543+0000 7fabc8000640 1 Processor -- start 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 -- start start 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 0x7fabc01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc0104f70 0x7fabc01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc01a0f50 con 0x7fabc01045b0 2024-03-11T19:20:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc01a10c0 con 0x7fabc0104f70 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc8000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc01a1280 con 0x7fabc01033c0 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 0x7fabc01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56668/0 (socket says 172.21.15.72:56668) 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 0x7fabc01999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40880/0 (socket says 172.21.15.72:40880) 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc5600640 1 -- 172.21.15.72:0/214902561 learned_addr learned my addr 172.21.15.72:0/214902561 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.544+0000 7fabc6000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc0104f70 0x7fabc01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 -- 172.21.15.72:0/214902561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc0104f70 msgr2=0x7fabc01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc0104f70 0x7fabc01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 -- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 msgr2=0x7fabc01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 0x7fabc01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 -- 172.21.15.72:0/214902561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009560 con 0x7fabc01033c0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc5600640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabb800ebc0 tx=0x7fabb800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8003b70 con 0x7fabc01033c0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb800eec0 con 0x7fabc01033c0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8019800 con 0x7fabc01033c0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.545+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc01a1560 con 0x7fabc01033c0 2024-03-11T19:20:02.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.546+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc01a1ac0 con 0x7fabc01033c0 2024-03-11T19:20:02.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.547+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc010a610 con 0x7fabc01033c0 2024-03-11T19:20:02.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.548+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fabb8004b60 con 0x7fabc01033c0 2024-03-11T19:20:02.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.550+0000 7fabbe000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 0x7faba80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.550+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fabb8014070 con 0x7fabc01033c0 2024-03-11T19:20:02.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.550+0000 7fabc4c00640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 0x7faba80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.551+0000 7fabc4c00640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 0x7faba80a3f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fabc019a970 tx=0x7fabac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.552+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabb8096d80 con 0x7fabc01033c0 2024-03-11T19:20:02.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:02 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1838631438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:02.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:02 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2786210589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:20:02.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.749+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabc010ee20 con 0x7fabc01033c0 2024-03-11T19:20:02.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.750+0000 7fabbe000640 1 -- 172.21.15.72:0/214902561 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabb809a440 con 0x7fabc01033c0 2024-03-11T19:20:02.754 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 msgr2=0x7faba80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 0x7faba80a3f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fabc019a970 tx=0x7fabac005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 msgr2=0x7fabc0199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fabb800ebc0 tx=0x7fabb800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 shutdown_connections 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7faba80a1ab0 0x7faba80a3f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabc0104f70 0x7fabc01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabc01045b0 0x7fabc01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 --2- 172.21.15.72:0/214902561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabc01033c0 0x7fabc0199460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 >> 172.21.15.72:0/214902561 conn(0x7fabc00fe930 msgr2=0x7fabc01077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:02.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 shutdown_connections 2024-03-11T19:20:02.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.755+0000 7fabc8000640 1 -- 172.21.15.72:0/214902561 wait complete. 2024-03-11T19:20:02.779 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 -- 172.21.15.72:0/3698199385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a0104590 msgr2=0x7f25a0104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 --2- 172.21.15.72:0/3698199385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a0104590 0x7f25a0104a10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f258c009900 tx=0x7f258c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 -- 172.21.15.72:0/3698199385 shutdown_connections 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 --2- 172.21.15.72:0/3698199385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 0x7f25a010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 --2- 172.21.15.72:0/3698199385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a0104590 0x7f25a0104a10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 --2- 172.21.15.72:0/3698199385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a01033a0 0x7f25a01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 -- 172.21.15.72:0/3698199385 >> 172.21.15.72:0/3698199385 conn(0x7f25a00fe930 msgr2=0x7f25a0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:02.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 -- 172.21.15.72:0/3698199385 shutdown_connections 2024-03-11T19:20:02.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.985+0000 7f25a8800640 1 -- 172.21.15.72:0/3698199385 wait complete. 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.986+0000 7f25a8800640 1 Processor -- start 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.986+0000 7f25a8800640 1 -- start start 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 0x7f25a0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 0x7f25a0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 0x7f25a01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01a0f40 con 0x7f25a01033a0 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01a10b0 con 0x7f25a0104f50 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a8800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01a1270 con 0x7f25a0104590 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 0x7f25a0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 0x7f25a0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 0x7f25a01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 0x7f25a0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40894/0 (socket says 172.21.15.72:40894) 2024-03-11T19:20:02.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a5e00640 1 -- 172.21.15.72:0/2814850266 learned_addr learned my addr 172.21.15.72:0/2814850266 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.987+0000 7f25a5400640 1 -- 172.21.15.72:0/2814850266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 msgr2=0x7f25a01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a5400640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 0x7f25a01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a5400640 1 -- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 msgr2=0x7f25a0199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a5400640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 0x7f25a0199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a5400640 1 -- 172.21.15.72:0/2814850266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c009840 con 0x7f25a0104590 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a5400640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 0x7f25a0199990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f258c02c170 tx=0x7f258c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0043d0 con 0x7f25a0104590 2024-03-11T19:20:02.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c009560 con 0x7f25a0104590 2024-03-11T19:20:02.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f258c0052d0 con 0x7f25a0104590 2024-03-11T19:20:02.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c003c40 con 0x7f25a0104590 2024-03-11T19:20:02.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.988+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a01a1840 con 0x7f25a0104590 2024-03-11T19:20:02.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.989+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a010a6c0 con 0x7f25a0104590 2024-03-11T19:20:02.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.991+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f258c005440 con 0x7f25a0104590 2024-03-11T19:20:02.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.993+0000 7f259aa00640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 0x7f25880a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:02.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.993+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f258c0f0420 con 0x7f25a0104590 2024-03-11T19:20:02.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.993+0000 7f25a5e00640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 0x7f25880a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:02.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.993+0000 7f25a5e00640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 0x7f25880a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25940058c0 tx=0x7f2594005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:02.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:02.995+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f258c0b9390 con 0x7f25a0104590 2024-03-11T19:20:03.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.192+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f25a010f050 con 0x7f25a0104590 2024-03-11T19:20:03.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.193+0000 7f259aa00640 1 -- 172.21.15.72:0/2814850266 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f258c0430a0 con 0x7f25a0104590 2024-03-11T19:20:03.197 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 msgr2=0x7f25880a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 0x7f25880a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25940058c0 tx=0x7f2594005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 msgr2=0x7f25a0199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 0x7f25a0199990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f258c02c170 tx=0x7f258c005e80 comp rx=0 tx=0).stop 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 shutdown_connections 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25880a1a30 0x7f25880a3ef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25a0104f50 0x7f25a01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a0104590 0x7f25a0199990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 --2- 172.21.15.72:0/2814850266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25a01033a0 0x7f25a0199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 >> 172.21.15.72:0/2814850266 conn(0x7f25a00fe930 msgr2=0x7f25a0107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:03.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 shutdown_connections 2024-03-11T19:20:03.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.196+0000 7f25a8800640 1 -- 172.21.15.72:0/2814850266 wait complete. 2024-03-11T19:20:03.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:20:03.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:03 smithi072 ceph-mon[31695]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:03.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:03 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/451345756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:03 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/214902561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 -- 172.21.15.72:0/3710483699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 msgr2=0x7f9e50107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 --2- 172.21.15.72:0/3710483699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e50107bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009900 tx=0x7f9e4402bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 -- 172.21.15.72:0/3710483699 shutdown_connections 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 --2- 172.21.15.72:0/3710483699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e5010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 --2- 172.21.15.72:0/3710483699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 0x7f9e50108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 --2- 172.21.15.72:0/3710483699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e50107bc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.428+0000 7f9e58600640 1 -- 172.21.15.72:0/3710483699 >> 172.21.15.72:0/3710483699 conn(0x7f9e50076740 msgr2=0x7f9e50078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.429+0000 7f9e58600640 1 -- 172.21.15.72:0/3710483699 shutdown_connections 2024-03-11T19:20:03.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.429+0000 7f9e58600640 1 -- 172.21.15.72:0/3710483699 wait complete. 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 Processor -- start 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 -- start start 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e5019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 0x7f9e5019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501a5370 con 0x7f9e50109350 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501a54e0 con 0x7f9e50108990 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e58600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501a56a0 con 0x7f9e501077a0 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e56600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:03.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e56600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40900/0 (socket says 172.21.15.72:40900) 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e56600640 1 -- 172.21.15.72:0/3469254950 learned_addr learned my addr 172.21.15.72:0/3469254950 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e55200640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 0x7f9e5019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.430+0000 7f9e55c00640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e5019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 -- 172.21.15.72:0/3469254950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 msgr2=0x7f9e5019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e5019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 -- 172.21.15.72:0/3469254950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 msgr2=0x7f9e5019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 0x7f9e5019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 -- 172.21.15.72:0/3469254950 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e48009840 con 0x7f9e50109350 2024-03-11T19:20:03.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e56600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800efd0 tx=0x7f9e4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:03.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48009b40 con 0x7f9e50109350 2024-03-11T19:20:03.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.431+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e44009560 con 0x7f9e50109350 2024-03-11T19:20:03.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.432+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e48003b70 con 0x7f9e50109350 2024-03-11T19:20:03.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.432+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480108f0 con 0x7f9e50109350 2024-03-11T19:20:03.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.432+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e501a5d00 con 0x7f9e50109350 2024-03-11T19:20:03.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.433+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5010eac0 con 0x7f9e50109350 2024-03-11T19:20:03.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.435+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9e48003ce0 con 0x7f9e50109350 2024-03-11T19:20:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:03 smithi143 ceph-mon[36013]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:03 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/451345756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:03 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/214902561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.436+0000 7f9e4e000640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 0x7f9e380a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:03.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.436+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9e48014070 con 0x7f9e50109350 2024-03-11T19:20:03.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.436+0000 7f9e55c00640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 0x7f9e380a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:03.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.437+0000 7f9e55c00640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 0x7f9e380a3ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e4402c240 tx=0x7f9e4402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:03.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.441+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e48096f80 con 0x7f9e50109350 2024-03-11T19:20:03.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:03 smithi106 ceph-mon[35889]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:20:03.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:03 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/451345756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:03 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/214902561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:20:03.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.652+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e5010e530 con 0x7f9e50109350 2024-03-11T19:20:03.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.652+0000 7f9e4e000640 1 -- 172.21.15.72:0/3469254950 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+591 (secure 0 0 0) 0x7f9e4809a640 con 0x7f9e50109350 2024-03-11T19:20:03.656 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1618271736 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2331163134 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3661300568 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/618300141 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2881891200 2024-03-12T19:11:29.909764+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3531203560 2024-03-12T19:10:56.562024+0000 2024-03-11T19:20:03.657 INFO:teuthology.orchestra.run.smithi072.stderr:listed 10 entries 2024-03-11T19:20:03.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.655+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 msgr2=0x7f9e380a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.655+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 0x7f9e380a3ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e4402c240 tx=0x7f9e4402c7e0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.655+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 msgr2=0x7f9e501a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.655+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800efd0 tx=0x7f9e4800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 shutdown_connections 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9e380a1a30 0x7f9e380a3ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9e50109350 0x7f9e501a4e30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e50108990 0x7f9e5019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 --2- 172.21.15.72:0/3469254950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e501077a0 0x7f9e5019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 >> 172.21.15.72:0/3469254950 conn(0x7f9e50076740 msgr2=0x7f9e5010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 shutdown_connections 2024-03-11T19:20:03.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:03.656+0000 7f9e58600640 1 -- 172.21.15.72:0/3469254950 wait complete. 2024-03-11T19:20:03.677 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi072.front.sepia.ceph.com... 2024-03-11T19:20:03.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:20:03.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:20:03.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi072.stderr:ceph-fuse[75636]: fuse finished with error 0 and tester_r 0 2024-03-11T19:20:03.768 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T19:20:04.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2814850266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:04.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3469254950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2814850266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:04.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3469254950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:04.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2814850266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:20:04.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3469254950' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:20:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:05 smithi072 ceph-mon[31695]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:05 smithi143 ceph-mon[36013]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:05.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:05 smithi106 ceph-mon[35889]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:07 smithi143 ceph-mon[36013]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:07 smithi106 ceph-mon[35889]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:07.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:07 smithi072 ceph-mon[31695]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:20:09.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:09 smithi143 ceph-mon[36013]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:09.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:09 smithi106 ceph-mon[35889]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:09.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:09 smithi072 ceph-mon[31695]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:09.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:20:09.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:20:09.871 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-11T19:20:09.939 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:20:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 -- 172.21.15.72:0/2046504289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 msgr2=0x7f09a4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 --2- 172.21.15.72:0/2046504289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a4106940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f098c009900 tx=0x7f098c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 -- 172.21.15.72:0/2046504289 shutdown_connections 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 --2- 172.21.15.72:0/2046504289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 0x7f09a41092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 --2- 172.21.15.72:0/2046504289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a4106940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 --2- 172.21.15.72:0/2046504289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 -- 172.21.15.72:0/2046504289 >> 172.21.15.72:0/2046504289 conn(0x7f09a40fb610 msgr2=0x7f09a40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 -- 172.21.15.72:0/2046504289 shutdown_connections 2024-03-11T19:20:10.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.155+0000 7f09aae00640 1 -- 172.21.15.72:0/2046504289 wait complete. 2024-03-11T19:20:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.156+0000 7f09aae00640 1 Processor -- start 2024-03-11T19:20:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.156+0000 7f09aae00640 1 -- start start 2024-03-11T19:20:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.156+0000 7f09aae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a41994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.156+0000 7f09aae00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09aae00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 0x7f09a41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09aae00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41a0f90 con 0x7f09a4104550 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09aae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41a1100 con 0x7f09a4101c20 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09aae00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41a12c0 con 0x7f09a4106e80 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40922/0 (socket says 172.21.15.72:40922) 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 -- 172.21.15.72:0/803427619 learned_addr learned my addr 172.21.15.72:0/803427619 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09a3e00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a41994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09a8e00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 0x7f09a41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 -- 172.21.15.72:0/803427619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 msgr2=0x7f09a41a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 0x7f09a41a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 -- 172.21.15.72:0/803427619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 msgr2=0x7f09a41994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a41994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 -- 172.21.15.72:0/803427619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009560 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f09a3e00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a41994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.157+0000 7f099b400640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f098c000c00 tx=0x7f098c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.158+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c004df0 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.158+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f098c005430 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.158+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c03e450 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.158+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a41a1540 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.158+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41a1aa0 con 0x7f09a4104550 2024-03-11T19:20:10.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.159+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a41089d0 con 0x7f09a4104550 2024-03-11T19:20:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.161+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f098c0055a0 con 0x7f09a4104550 2024-03-11T19:20:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.162+0000 7f09a1600640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 0x7f09880a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.163+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f098c0f0120 con 0x7f09a4104550 2024-03-11T19:20:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.163+0000 7f09a3e00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 0x7f09880a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.163+0000 7f09a3e00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 0x7f09880a3f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0994005fd0 tx=0x7f0994009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.165+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f098c0b9090 con 0x7f09a4104550 2024-03-11T19:20:10.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.364+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a410ce00 con 0x7f09a4104550 2024-03-11T19:20:10.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.368+0000 7f09a1600640 1 -- 172.21.15.72:0/803427619 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7f098c0bc750 con 0x7f09a4104550 2024-03-11T19:20:10.372 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:10.372 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 msgr2=0x7f09880a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 0x7f09880a3f70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0994005fd0 tx=0x7f0994009600 comp rx=0 tx=0).stop 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 msgr2=0x7f09a41999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f098c000c00 tx=0x7f098c004270 comp rx=0 tx=0).stop 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 shutdown_connections 2024-03-11T19:20:10.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f09880a1ab0 0x7f09880a3f70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09a4106e80 0x7f09a41a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09a4104550 0x7f09a41999e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 --2- 172.21.15.72:0/803427619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09a4101c20 0x7f09a41994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 >> 172.21.15.72:0/803427619 conn(0x7f09a40fb610 msgr2=0x7f09a4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 shutdown_connections 2024-03-11T19:20:10.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.371+0000 7f09aae00640 1 -- 172.21.15.72:0/803427619 wait complete. 2024-03-11T19:20:10.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:20:10.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 -- 172.21.15.72:0/2715709198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301045b0 msgr2=0x7f4330104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 --2- 172.21.15.72:0/2715709198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301045b0 0x7f4330104a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f431c009900 tx=0x7f431c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:10.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 -- 172.21.15.72:0/2715709198 shutdown_connections 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 --2- 172.21.15.72:0/2715709198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f433010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 --2- 172.21.15.72:0/2715709198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301045b0 0x7f4330104a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 --2- 172.21.15.72:0/2715709198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301033c0 0x7f43301037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 -- 172.21.15.72:0/2715709198 >> 172.21.15.72:0/2715709198 conn(0x7f43300fe970 msgr2=0x7f4330100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 -- 172.21.15.72:0/2715709198 shutdown_connections 2024-03-11T19:20:10.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.603+0000 7f4337e00640 1 -- 172.21.15.72:0/2715709198 wait complete. 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.604+0000 7f4337e00640 1 Processor -- start 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.604+0000 7f4337e00640 1 -- start start 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301033c0 0x7f4330199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 0x7f4330199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301a0f10 con 0x7f43301045b0 2024-03-11T19:20:10.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301a1080 con 0x7f43301033c0 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4337e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301a1240 con 0x7f4330104f70 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4334a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 0x7f4330199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56742/0 (socket says 172.21.15.72:56742) 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 -- 172.21.15.72:0/2692016851 learned_addr learned my addr 172.21.15.72:0/2692016851 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4334a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 0x7f4330199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40932/0 (socket says 172.21.15.72:40932) 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335400640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301033c0 0x7f4330199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 -- 172.21.15.72:0/2692016851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301033c0 msgr2=0x7f4330199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301033c0 0x7f4330199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 -- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 msgr2=0x7f4330199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 0x7f4330199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.605+0000 7f4335e00640 1 -- 172.21.15.72:0/2692016851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c009560 con 0x7f4330104f70 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f4335e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f432800eb10 tx=0x7f432800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:10.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328004b20 con 0x7f4330104f70 2024-03-11T19:20:10.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4328004c80 con 0x7f4330104f70 2024-03-11T19:20:10.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43301a1520 con 0x7f4330104f70 2024-03-11T19:20:10.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328010670 con 0x7f4330104f70 2024-03-11T19:20:10.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.606+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43301a1a00 con 0x7f4330104f70 2024-03-11T19:20:10.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.607+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433010a760 con 0x7f4330104f70 2024-03-11T19:20:10.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.609+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4328005020 con 0x7f4330104f70 2024-03-11T19:20:10.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.610+0000 7f432e000640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 0x7f43180a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:10.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.611+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4328014070 con 0x7f4330104f70 2024-03-11T19:20:10.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.611+0000 7f4335400640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 0x7f43180a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:10.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.612+0000 7f4335400640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 0x7f43180a3ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4330104410 tx=0x7f4324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:10.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.615+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4328096b50 con 0x7f4330104f70 2024-03-11T19:20:10.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.814+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f433010edd0 con 0x7f4330104f70 2024-03-11T19:20:10.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.816+0000 7f432e000640 1 -- 172.21.15.72:0/2692016851 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7f432809a210 con 0x7f4330104f70 2024-03-11T19:20:10.820 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:10.820 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.819+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 msgr2=0x7f43180a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.819+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 0x7f43180a3ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4330104410 tx=0x7f4324005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.819+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 msgr2=0x7f43301a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.819+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f432800eb10 tx=0x7f432800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 shutdown_connections 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43180a1a30 0x7f43180a3ef0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4330104f70 0x7f43301a09d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43301045b0 0x7f4330199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 --2- 172.21.15.72:0/2692016851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43301033c0 0x7f4330199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 >> 172.21.15.72:0/2692016851 conn(0x7f43300fe970 msgr2=0x7f4330107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 shutdown_connections 2024-03-11T19:20:10.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:10.820+0000 7f4337e00640 1 -- 172.21.15.72:0/2692016851 wait complete. 2024-03-11T19:20:10.843 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-11T19:20:11.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 -- 172.21.15.72:0/4089131074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc1077a0 msgr2=0x7f65dc107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:11.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 --2- 172.21.15.72:0/4089131074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc1077a0 0x7f65dc107bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009900 tx=0x7f65cc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:11.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 -- 172.21.15.72:0/4089131074 shutdown_connections 2024-03-11T19:20:11.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 --2- 172.21.15.72:0/4089131074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc109350 0x7f65dc10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:11.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 --2- 172.21.15.72:0/4089131074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 0x7f65dc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:11.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 --2- 172.21.15.72:0/4089131074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc1077a0 0x7f65dc107bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:11.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 -- 172.21.15.72:0/4089131074 >> 172.21.15.72:0/4089131074 conn(0x7f65dc076740 msgr2=0x7f65dc078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:11.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 -- 172.21.15.72:0/4089131074 shutdown_connections 2024-03-11T19:20:11.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.054+0000 7f65e1a00640 1 -- 172.21.15.72:0/4089131074 wait complete. 2024-03-11T19:20:11.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 Processor -- start 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 -- start start 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 0x7f65dc19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 0x7f65dc1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.055+0000 7f65e1a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a53a0 con 0x7f65dc109350 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65e1a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a5510 con 0x7f65dc1077a0 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65e1a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a56d0 con 0x7f65dc108990 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65db400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 0x7f65dc1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65d3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 0x7f65dc19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65db400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 0x7f65dc1a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40950/0 (socket says 172.21.15.72:40950) 2024-03-11T19:20:11.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:58732/0 (socket says 172.21.15.72:58732) 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65db400640 1 -- 172.21.15.72:0/242525636 learned_addr learned my addr 172.21.15.72:0/242525636 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 -- 172.21.15.72:0/242525636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 msgr2=0x7f65dc19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 0x7f65dc19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 -- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 msgr2=0x7f65dc1a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 0x7f65dc1a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.056+0000 7f65daa00640 1 -- 172.21.15.72:0/242525636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009560 con 0x7f65dc1077a0 2024-03-11T19:20:11.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65daa00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65cc036040 tx=0x7f65cc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc004aa0 con 0x7f65dc1077a0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65cc0052d0 con 0x7f65dc1077a0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc003dc0 con 0x7f65dc1077a0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc1a5950 con 0x7f65dc1077a0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.057+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1a5de0 con 0x7f65dc1077a0 2024-03-11T19:20:11.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.058+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc10eb90 con 0x7f65dc1077a0 2024-03-11T19:20:11.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.060+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f65cc005440 con 0x7f65dc1077a0 2024-03-11T19:20:11.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.062+0000 7f65d3400640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 0x7f65bc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:11.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.062+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f65cc0effa0 con 0x7f65dc1077a0 2024-03-11T19:20:11.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.062+0000 7f65d3e00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 0x7f65bc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:11.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.063+0000 7f65d3e00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 0x7f65bc0a3f70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65dc19edc0 tx=0x7f65c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:11.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.064+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65cc0b8f10 con 0x7f65dc1077a0 2024-03-11T19:20:11.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:11.299+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f65dc02ce40 con 0x7f65dc1077a0 2024-03-11T19:20:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:11 smithi106 ceph-mon[35889]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:11 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/803427619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:11 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2692016851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:11 smithi072 ceph-mon[31695]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:11 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/803427619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:11 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2692016851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:11 smithi143 ceph-mon[36013]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:11 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/803427619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:11 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2692016851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:12.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.204+0000 7f65d3400640 1 -- 172.21.15.72:0/242525636 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f65cc0430b0 con 0x7f65dc1077a0 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.207+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 msgr2=0x7f65bc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.207+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 0x7f65bc0a3f70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65dc19edc0 tx=0x7f65c4005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.207+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 msgr2=0x7f65dc19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.207+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65cc036040 tx=0x7f65cc004270 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 shutdown_connections 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f65bc0a1ab0 0x7f65bc0a3f70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65dc109350 0x7f65dc1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc108990 0x7f65dc19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 --2- 172.21.15.72:0/242525636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65dc1077a0 0x7f65dc19d8b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 >> 172.21.15.72:0/242525636 conn(0x7f65dc076740 msgr2=0x7f65dc10bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 shutdown_connections 2024-03-11T19:20:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.208+0000 7f65e1a00640 1 -- 172.21.15.72:0/242525636 wait complete. 2024-03-11T19:20:12.231 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-11T19:20:12.231 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-11T19:20:12.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 -- 172.21.15.72:0/2454748808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4668108990 msgr2=0x7f4668108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 --2- 172.21.15.72:0/2454748808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4668108990 0x7f4668108e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4650009900 tx=0x7f465002bc50 comp rx=0 tx=0).stop 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 -- 172.21.15.72:0/2454748808 shutdown_connections 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 --2- 172.21.15.72:0/2454748808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668109350 0x7f466810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 --2- 172.21.15.72:0/2454748808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4668108990 0x7f4668108e10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 --2- 172.21.15.72:0/2454748808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46681077a0 0x7f4668107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 -- 172.21.15.72:0/2454748808 >> 172.21.15.72:0/2454748808 conn(0x7f4668076740 msgr2=0x7f4668078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 -- 172.21.15.72:0/2454748808 shutdown_connections 2024-03-11T19:20:12.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.437+0000 7f466ee00640 1 -- 172.21.15.72:0/2454748808 wait complete. 2024-03-11T19:20:12.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.438+0000 7f466ee00640 1 Processor -- start 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.438+0000 7f466ee00640 1 -- start start 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 0x7f466819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4668109350 0x7f46681a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681a5340 con 0x7f46681077a0 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681a54b0 con 0x7f4668109350 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681a5670 con 0x7f4668108990 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39674/0 (socket says 172.21.15.72:39674) 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 -- 172.21.15.72:0/3147143979 learned_addr learned my addr 172.21.15.72:0/3147143979 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f465fe00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 0x7f466819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f466ce00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4668109350 0x7f46681a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:12.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 -- 172.21.15.72:0/3147143979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 msgr2=0x7f466819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 0x7f466819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 -- 172.21.15.72:0/3147143979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4668109350 msgr2=0x7f46681a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4668109350 0x7f46681a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f4667e00640 1 -- 172.21.15.72:0/3147143979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658009490 con 0x7f46681077a0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.439+0000 7f465fe00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 0x7f466819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f4667e00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f465800e9c0 tx=0x7f465800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800cde0 con 0x7f46681077a0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4658010910 con 0x7f46681077a0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658018a10 con 0x7f46681077a0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4650009560 con 0x7f46681077a0 2024-03-11T19:20:12.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.440+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46681a5c50 con 0x7f46681077a0 2024-03-11T19:20:12.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.441+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466810eac0 con 0x7f46681077a0 2024-03-11T19:20:12.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.443+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4658010a80 con 0x7f46681077a0 2024-03-11T19:20:12.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.444+0000 7f4665600640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 0x7f464c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:12.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.445+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4658014070 con 0x7f46681077a0 2024-03-11T19:20:12.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.445+0000 7f465fe00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 0x7f464c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:12.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.445+0000 7f465fe00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 0x7f464c0a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4650009230 tx=0x7f4650005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:12.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.449+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4658096ce0 con 0x7f46681077a0 2024-03-11T19:20:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:12 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/242525636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:12 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:12 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/242525636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:12 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:12 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/242525636' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:12 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:20:12.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:12.686+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f466802ce70 con 0x7f46681077a0 2024-03-11T19:20:13.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.204+0000 7f4665600640 1 -- 172.21.15.72:0/3147143979 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f465809a3a0 con 0x7f46681077a0 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 msgr2=0x7f464c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 0x7f464c0a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4650009230 tx=0x7f4650005fd0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 msgr2=0x7f466819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f465800e9c0 tx=0x7f465800ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 shutdown_connections 2024-03-11T19:20:13.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f464c0a1a30 0x7f464c0a3ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4668109350 0x7f46681a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4668108990 0x7f466819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 --2- 172.21.15.72:0/3147143979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46681077a0 0x7f466819d850 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 >> 172.21.15.72:0/3147143979 conn(0x7f4668076740 msgr2=0x7f466810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 shutdown_connections 2024-03-11T19:20:13.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.208+0000 7f466ee00640 1 -- 172.21.15.72:0/3147143979 wait complete. 2024-03-11T19:20:13.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.423+0000 7f3e2f200640 1 -- 172.21.15.72:0/1573304619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 msgr2=0x7f3e301094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.423+0000 7f3e2f200640 1 --2- 172.21.15.72:0/1573304619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301094a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009ca0 tx=0x7f3e2802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 -- 172.21.15.72:0/1573304619 shutdown_connections 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 --2- 172.21.15.72:0/1573304619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301094a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 --2- 172.21.15.72:0/1573304619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 0x7f3e301026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 --2- 172.21.15.72:0/1573304619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e30101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 -- 172.21.15.72:0/1573304619 >> 172.21.15.72:0/1573304619 conn(0x7f3e300fc5f0 msgr2=0x7f3e300fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 -- 172.21.15.72:0/1573304619 shutdown_connections 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.424+0000 7f3e2f200640 1 -- 172.21.15.72:0/1573304619 wait complete. 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 Processor -- start 2024-03-11T19:20:13.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 -- start start 2024-03-11T19:20:13.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e301992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:13.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 0x7f3e301997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a0da0 con 0x7f3e30102c10 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a0f10 con 0x7f3e30101060 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.425+0000 7f3e2f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e301a10d0 con 0x7f3e30102250 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e301992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 0x7f3e301997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39688/0 (socket says 172.21.15.72:39688) 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 -- 172.21.15.72:0/2176891552 learned_addr learned my addr 172.21.15.72:0/2176891552 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 -- 172.21.15.72:0/2176891552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 msgr2=0x7f3e301997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 0x7f3e301997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 -- 172.21.15.72:0/2176891552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 msgr2=0x7f3e301992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:13.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e301992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2e800640 1 -- 172.21.15.72:0/2176891552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e28009900 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.426+0000 7f3e2de00640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e301992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e2e800640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e2802c630 tx=0x7f3e28032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28034e20 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e28003e50 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e301a1350 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004810 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.427+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e301a1850 con 0x7f3e30102c10 2024-03-11T19:20:13.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.428+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3010ca40 con 0x7f3e30102c10 2024-03-11T19:20:13.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.430+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3e2800a040 con 0x7f3e30102c10 2024-03-11T19:20:13.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.431+0000 7f3e26a00640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 0x7f3e140a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:13.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.432+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3e280f0080 con 0x7f3e30102c10 2024-03-11T19:20:13.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.432+0000 7f3e2de00640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 0x7f3e140a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:13.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.433+0000 7f3e2de00640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 0x7f3e140a3f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e301020b0 tx=0x7f3e20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:13.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.436+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e280b8ff0 con 0x7f3e30102c10 2024-03-11T19:20:13.591 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:13 smithi143 ceph-mon[36013]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:13.591 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:13 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:20:13.591 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:13 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:13.591 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:13 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:20:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:13 smithi106 ceph-mon[35889]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:13 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:20:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:13 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:13 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:20:13.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:13.664+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f3e3010cc00 con 0x7f3e30102c10 2024-03-11T19:20:13.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:13 smithi072 ceph-mon[31695]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:20:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:13 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:20:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:13 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:13 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:20:14.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.216+0000 7f3e26a00640 1 -- 172.21.15.72:0/2176891552 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3e280bc6b0 con 0x7f3e30102c10 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 msgr2=0x7f3e140a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 0x7f3e140a3f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e301020b0 tx=0x7f3e20005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 msgr2=0x7f3e301a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e2802c630 tx=0x7f3e28032f80 comp rx=0 tx=0).stop 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 shutdown_connections 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3e140a1ab0 0x7f3e140a3f70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e30102c10 0x7f3e301a0860 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e30102250 0x7f3e301997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 --2- 172.21.15.72:0/2176891552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e30101060 0x7f3e301992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 >> 172.21.15.72:0/2176891552 conn(0x7f3e300fc5f0 msgr2=0x7f3e300733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 shutdown_connections 2024-03-11T19:20:14.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.220+0000 7f3e2f200640 1 -- 172.21.15.72:0/2176891552 wait complete. 2024-03-11T19:20:14.249 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-11T19:20:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 -- 172.21.15.72:0/867596318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490108990 msgr2=0x7f6490108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 --2- 172.21.15.72:0/867596318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490108990 0x7f6490108e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6478009860 tx=0x7f647802bb70 comp rx=0 tx=0).stop 2024-03-11T19:20:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 -- 172.21.15.72:0/867596318 shutdown_connections 2024-03-11T19:20:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 --2- 172.21.15.72:0/867596318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6490109350 0x7f649010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 --2- 172.21.15.72:0/867596318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490108990 0x7f6490108e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 --2- 172.21.15.72:0/867596318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f64901077a0 0x7f6490107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 -- 172.21.15.72:0/867596318 >> 172.21.15.72:0/867596318 conn(0x7f6490076740 msgr2=0x7f6490078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 -- 172.21.15.72:0/867596318 shutdown_connections 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.438+0000 7f6496c00640 1 -- 172.21.15.72:0/867596318 wait complete. 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 Processor -- start 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 -- start start 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 0x7f649019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490109350 0x7f64901a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a5340 con 0x7f64901077a0 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a54b0 con 0x7f6490108990 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.439+0000 7f6496c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a5670 con 0x7f6490109350 2024-03-11T19:20:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39708/0 (socket says 172.21.15.72:39708) 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 -- 172.21.15.72:0/1031004644 learned_addr learned my addr 172.21.15.72:0/1031004644 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f6494c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490109350 0x7f64901a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f6487400640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 0x7f649019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 -- 172.21.15.72:0/1031004644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490109350 msgr2=0x7f64901a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490109350 0x7f64901a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 -- 172.21.15.72:0/1031004644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 msgr2=0x7f649019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 0x7f649019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 -- 172.21.15.72:0/1031004644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488009900 con 0x7f64901077a0 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f6487400640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 0x7f649019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:20:14.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648fe00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f648000e9c0 tx=0x7f648000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.440+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800096e0 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.441+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6478009490 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.441+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f648000cd80 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.441+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480010430 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.441+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64901a5c50 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.442+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649010eb40 con 0x7f64901077a0 2024-03-11T19:20:14.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.444+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6480003680 con 0x7f64901077a0 2024-03-11T19:20:14.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.445+0000 7f648d600640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 0x7f64700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:14.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.445+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6480014070 con 0x7f64901077a0 2024-03-11T19:20:14.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.445+0000 7f6487400640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 0x7f64700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:14.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.446+0000 7f6487400640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 0x7f64700a3ef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f647802c180 tx=0x7f6478034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:14.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.449+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6480096b30 con 0x7f64901077a0 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-11T19:20:14.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:14 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:20:14.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:14.669+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f649002ce40 con 0x7f64901077a0 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-11T19:20:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:14 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:20:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:20:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3147143979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:20:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:20:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:20:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-11T19:20:14.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-11T19:20:14.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:14 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:20:15.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.227+0000 7f648d600640 1 -- 172.21.15.72:0/1031004644 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f648009a1f0 con 0x7f64901077a0 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 msgr2=0x7f64700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 0x7f64700a3ef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f647802c180 tx=0x7f6478034be0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 msgr2=0x7f649019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f648000e9c0 tx=0x7f648000ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 shutdown_connections 2024-03-11T19:20:15.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f64700a1a30 0x7f64700a3ef0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6490109350 0x7f64901a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6490108990 0x7f649019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 --2- 172.21.15.72:0/1031004644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64901077a0 0x7f649019d850 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 >> 172.21.15.72:0/1031004644 conn(0x7f6490076740 msgr2=0x7f649010bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 shutdown_connections 2024-03-11T19:20:15.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.231+0000 7f6496c00640 1 -- 172.21.15.72:0/1031004644 wait complete. 2024-03-11T19:20:15.259 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-11T19:20:15.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 -- 172.21.15.72:0/4031099190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c108f10 msgr2=0x7fce3c10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 --2- 172.21.15.72:0/4031099190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c108f10 0x7fce3c10b3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce34009be0 tx=0x7fce3402bf80 comp rx=0 tx=0).stop 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 -- 172.21.15.72:0/4031099190 shutdown_connections 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 --2- 172.21.15.72:0/4031099190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c108f10 0x7fce3c10b3a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 --2- 172.21.15.72:0/4031099190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c073b60 0x7fce3c073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 --2- 172.21.15.72:0/4031099190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 -- 172.21.15.72:0/4031099190 >> 172.21.15.72:0/4031099190 conn(0x7fce3c0fc5f0 msgr2=0x7fce3c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 -- 172.21.15.72:0/4031099190 shutdown_connections 2024-03-11T19:20:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.470+0000 7fce42e00640 1 -- 172.21.15.72:0/4031099190 wait complete. 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.471+0000 7fce42e00640 1 Processor -- start 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.471+0000 7fce42e00640 1 -- start start 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c073b60 0x7fce3c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c108f10 0x7fce3c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a0d50 con 0x7fce3c108f10 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a0ec0 con 0x7fce3c075140 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1a1080 con 0x7fce3c073b60 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:46998/0 (socket says 172.21.15.72:46998) 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 -- 172.21.15.72:0/462234883 learned_addr learned my addr 172.21.15.72:0/462234883 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce42400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c108f10 0x7fce3c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41a00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c073b60 0x7fce3c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 -- 172.21.15.72:0/462234883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c073b60 msgr2=0x7fce3c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c073b60 0x7fce3c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 -- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c108f10 msgr2=0x7fce3c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c108f10 0x7fce3c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.472+0000 7fce41000640 1 -- 172.21.15.72:0/462234883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009840 con 0x7fce3c075140 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce41000640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce2800e9c0 tx=0x7fce2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800ccc0 con 0x7fce3c075140 2024-03-11T19:20:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c1a1360 con 0x7fce3c075140 2024-03-11T19:20:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce2800ce20 con 0x7fce3c075140 2024-03-11T19:20:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28010430 con 0x7fce3c075140 2024-03-11T19:20:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.473+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c1a18c0 con 0x7fce3c075140 2024-03-11T19:20:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.474+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c107cd0 con 0x7fce3c075140 2024-03-11T19:20:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.476+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fce28004f80 con 0x7fce3c075140 2024-03-11T19:20:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.477+0000 7fce3a000640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 0x7fce240a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.477+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fce28014070 con 0x7fce3c075140 2024-03-11T19:20:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.477+0000 7fce41a00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 0x7fce240a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.478+0000 7fce41a00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 0x7fce240a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce30009bd0 tx=0x7fce30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:15.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.480+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce28096b10 con 0x7fce3c075140 2024-03-11T19:20:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: pgmap v342: 129 pgs: 129 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-11T19:20:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:20:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:active 2024-03-11T19:20:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:20:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:20:15.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:20:15.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-11T19:20:15.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:20:15.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:20:15.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:15 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:20:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: pgmap v342: 129 pgs: 129 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:active 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:20:15.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:20:15.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: pgmap v342: 129 pgs: 129 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-11T19:20:15.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2176891552' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:20:15.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] up:active 2024-03-11T19:20:15.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:20:15.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:15 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:20:15.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.707+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fce3c02ce10 con 0x7fce240a1a30 2024-03-11T19:20:15.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.738+0000 7fce3a000640 1 -- 172.21.15.72:0/462234883 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fce3c02ce10 con 0x7fce240a1a30 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 msgr2=0x7fce240a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 0x7fce240a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce30009bd0 tx=0x7fce30005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 msgr2=0x7fce3c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce2800e9c0 tx=0x7fce2800ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 shutdown_connections 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fce240a1a30 0x7fce240a3ef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce3c108f10 0x7fce3c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c075140 0x7fce3c1997a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 --2- 172.21.15.72:0/462234883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce3c073b60 0x7fce3c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.740+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 >> 172.21.15.72:0/462234883 conn(0x7fce3c0fc5f0 msgr2=0x7fce3c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:15.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.741+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 shutdown_connections 2024-03-11T19:20:15.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.741+0000 7fce42e00640 1 -- 172.21.15.72:0/462234883 wait complete. 2024-03-11T19:20:15.762 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 -- 172.21.15.72:0/1753689874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368108990 msgr2=0x7f1368108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 --2- 172.21.15.72:0/1753689874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368108990 0x7f1368108e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1354009900 tx=0x7f135402bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 -- 172.21.15.72:0/1753689874 shutdown_connections 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 --2- 172.21.15.72:0/1753689874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1368109350 0x7f136810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 --2- 172.21.15.72:0/1753689874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368108990 0x7f1368108e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 --2- 172.21.15.72:0/1753689874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13681077a0 0x7f1368107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.960+0000 7f1370200640 1 -- 172.21.15.72:0/1753689874 >> 172.21.15.72:0/1753689874 conn(0x7f1368076740 msgr2=0x7f1368078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:15.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.961+0000 7f1370200640 1 -- 172.21.15.72:0/1753689874 shutdown_connections 2024-03-11T19:20:15.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.961+0000 7f1370200640 1 -- 172.21.15.72:0/1753689874 wait complete. 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.961+0000 7f1370200640 1 Processor -- start 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 -- start start 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 0x7f136819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368109350 0x7f13681a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13681a5360 con 0x7f1368109350 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13681a54d0 con 0x7f13681077a0 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f1370200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13681a5690 con 0x7f1368108990 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f136ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f136d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 0x7f136819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f136e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368109350 0x7f13681a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f136ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:35126/0 (socket says 172.21.15.72:35126) 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.962+0000 7f136ce00640 1 -- 172.21.15.72:0/4165812035 learned_addr learned my addr 172.21.15.72:0/4165812035 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 -- 172.21.15.72:0/4165812035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 msgr2=0x7f136819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 0x7f136819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 -- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368109350 msgr2=0x7f13681a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368109350 0x7f13681a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 -- 172.21.15.72:0/4165812035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c009490 con 0x7f1368108990 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136d800640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 0x7f136819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f136ce00640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f135402c6a0 tx=0x7f13540040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13540044c0 con 0x7f1368108990 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1354009560 con 0x7f1368108990 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13540052d0 con 0x7f1368108990 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354003e80 con 0x7f1368108990 2024-03-11T19:20:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.963+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a5ce0 con 0x7f1368108990 2024-03-11T19:20:15.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.964+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136810eac0 con 0x7f1368108990 2024-03-11T19:20:15.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.966+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1354005440 con 0x7f1368108990 2024-03-11T19:20:15.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.968+0000 7f1366000640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 0x7f13500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:15.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.968+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f13540f0220 con 0x7f1368108990 2024-03-11T19:20:15.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.968+0000 7f136d800640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 0x7f13500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:15.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.969+0000 7f136d800640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 0x7f13500a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13681087f0 tx=0x7f135c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:15.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:15.970+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13540b9290 con 0x7f1368108990 2024-03-11T19:20:16.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:20:15.716+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:20:16.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:20:15.716+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:20:16.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:20:15.716+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:20:16.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:20:15.716+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:20:16.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:20:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:20:15.716+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:20:16.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.198+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f136802ce10 con 0x7f13500a1a30 2024-03-11T19:20:16.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.231+0000 7f1366000640 1 -- 172.21.15.72:0/4165812035 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f136802ce10 con 0x7f13500a1a30 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 msgr2=0x7f13500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 0x7f13500a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13681087f0 tx=0x7f135c005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 msgr2=0x7f136819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f135402c6a0 tx=0x7f13540040c0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 shutdown_connections 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f13500a1a30 0x7f13500a3ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1368109350 0x7f13681a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.233+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1368108990 0x7f136819ddb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.234+0000 7f1370200640 1 --2- 172.21.15.72:0/4165812035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13681077a0 0x7f136819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.234+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 >> 172.21.15.72:0/4165812035 conn(0x7f1368076740 msgr2=0x7f136810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.234+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 shutdown_connections 2024-03-11T19:20:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.234+0000 7f1370200640 1 -- 172.21.15.72:0/4165812035 wait complete. 2024-03-11T19:20:16.258 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-11T19:20:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.449+0000 7f01efe00640 1 -- 172.21.15.72:0/4200058764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102250 msgr2=0x7f01f01026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.449+0000 7f01efe00640 1 --2- 172.21.15.72:0/4200058764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102250 0x7f01f01026d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009900 tx=0x7f01d802bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.449+0000 7f01efe00640 1 -- 172.21.15.72:0/4200058764 shutdown_connections 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.449+0000 7f01efe00640 1 --2- 172.21.15.72:0/4200058764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0102c10 0x7f01f01094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.449+0000 7f01efe00640 1 --2- 172.21.15.72:0/4200058764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102250 0x7f01f01026d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.450+0000 7f01efe00640 1 --2- 172.21.15.72:0/4200058764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0101060 0x7f01f0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.450+0000 7f01efe00640 1 -- 172.21.15.72:0/4200058764 >> 172.21.15.72:0/4200058764 conn(0x7f01f00fc5f0 msgr2=0x7f01f00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.450+0000 7f01efe00640 1 -- 172.21.15.72:0/4200058764 shutdown_connections 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.450+0000 7f01efe00640 1 -- 172.21.15.72:0/4200058764 wait complete. 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.450+0000 7f01efe00640 1 Processor -- start 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 -- start start 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 0x7f01f0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0102250 0x7f01f01998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a0e70 con 0x7f01f0102c10 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a0fe0 con 0x7f01f0101060 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01efe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a11a0 con 0x7f01f0102250 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01ef400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01ef400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39804/0 (socket says 172.21.15.72:39804) 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01ef400640 1 -- 172.21.15.72:0/2740646967 learned_addr learned my addr 172.21.15.72:0/2740646967 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01eea00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 0x7f01f0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.451+0000 7f01e6000640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0102250 0x7f01f01998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 -- 172.21.15.72:0/2740646967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0102250 msgr2=0x7f01f01998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0102250 0x7f01f01998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 -- 172.21.15.72:0/2740646967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 msgr2=0x7f01f0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 0x7f01f0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 -- 172.21.15.72:0/2740646967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8009840 con 0x7f01f0102c10 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01eea00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 0x7f01f0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01ef400640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f01e800efd0 tx=0x7f01e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8009b40 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01e8003b70 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d8009560 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e80108f0 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.452+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01a1780 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.453+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f010ca70 con 0x7f01f0102c10 2024-03-11T19:20:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.455+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f01e8003ce0 con 0x7f01f0102c10 2024-03-11T19:20:16.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.457+0000 7f01f4a00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 0x7f01d40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.458+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f01e8014070 con 0x7f01f0102c10 2024-03-11T19:20:16.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.458+0000 7f01eea00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 0x7f01d40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.458+0000 7f01eea00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 0x7f01d40a3f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01f01020b0 tx=0x7f01e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:16.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.462+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01e8096c00 con 0x7f01f0102c10 2024-03-11T19:20:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:16 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:20:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:16 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:active 2024-03-11T19:20:16.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:16 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:20:16.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:16 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:20:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:16 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:20:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:16 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:active 2024-03-11T19:20:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:16 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:20:16.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:16 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:20:16.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:16 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1031004644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:20:16.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:16 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] up:active 2024-03-11T19:20:16.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:16 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:20:16.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:16 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:20:16.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.687+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f01f0074ad0 con 0x7f01d40a1ac0 2024-03-11T19:20:16.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.697+0000 7f01f4a00640 1 -- 172.21.15.72:0/2740646967 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f01f0074ad0 con 0x7f01d40a1ac0 2024-03-11T19:20:16.700 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_0/68cfaaeb-0bac-46d7-b5d5-55eb3a056118 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 msgr2=0x7f01d40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 0x7f01d40a3f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01f01020b0 tx=0x7f01e0005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 msgr2=0x7f01f01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f01e800efd0 tx=0x7f01e800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 shutdown_connections 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f01d40a1ac0 0x7f01d40a3f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0102c10 0x7f01f01a0930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01f0102250 0x7f01f01998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 --2- 172.21.15.72:0/2740646967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0101060 0x7f01f0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.699+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 >> 172.21.15.72:0/2740646967 conn(0x7f01f00fc5f0 msgr2=0x7f01f00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.700+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 shutdown_connections 2024-03-11T19:20:16.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.700+0000 7f01efe00640 1 -- 172.21.15.72:0/2740646967 wait complete. 2024-03-11T19:20:16.721 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.923+0000 7ff3aaa00640 1 -- 172.21.15.72:0/866290007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a41076a0 msgr2=0x7ff3a4107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.923+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/866290007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a41076a0 0x7ff3a4107ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff398009900 tx=0x7ff39802bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 -- 172.21.15.72:0/866290007 shutdown_connections 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/866290007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 0x7ff3a410fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/866290007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a4108890 0x7ff3a4108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/866290007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a41076a0 0x7ff3a4107ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 -- 172.21.15.72:0/866290007 >> 172.21.15.72:0/866290007 conn(0x7ff3a4076650 msgr2=0x7ff3a4078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:16.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 -- 172.21.15.72:0/866290007 shutdown_connections 2024-03-11T19:20:16.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 -- 172.21.15.72:0/866290007 wait complete. 2024-03-11T19:20:16.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.924+0000 7ff3aaa00640 1 Processor -- start 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 -- start start 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a4108890 0x7ff3a419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 0x7ff3a41a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41a51f0 con 0x7ff3a41076a0 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41a5360 con 0x7ff3a4109250 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41a5520 con 0x7ff3a4108890 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3a9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3a9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39826/0 (socket says 172.21.15.72:39826) 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3a9600640 1 -- 172.21.15.72:0/2213418705 learned_addr learned my addr 172.21.15.72:0/2213418705 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3a8c00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a4108890 0x7ff3a419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.925+0000 7ff3aa000640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 0x7ff3a41a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 -- 172.21.15.72:0/2213418705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a4108890 msgr2=0x7ff3a419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a4108890 0x7ff3a419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 -- 172.21.15.72:0/2213418705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 msgr2=0x7ff3a41a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 0x7ff3a41a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 -- 172.21.15.72:0/2213418705 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390009490 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a9600640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff398005ab0 tx=0x7ff398005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980043d0 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff398009560 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3980052d0 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398003d40 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.926+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a41a5b70 con 0x7ff3a41076a0 2024-03-11T19:20:16.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.927+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a410ea40 con 0x7ff3a41076a0 2024-03-11T19:20:16.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.929+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff398005440 con 0x7ff3a41076a0 2024-03-11T19:20:16.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.930+0000 7ff3a2000640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 0x7ff38c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:16.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.931+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff3980ef840 con 0x7ff3a41076a0 2024-03-11T19:20:16.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.931+0000 7ff3a8c00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 0x7ff38c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:16.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.931+0000 7ff3a8c00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 0x7ff38c0a3f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3a419ec10 tx=0x7ff390005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:16.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:16.934+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3980b8760 con 0x7ff3a41076a0 2024-03-11T19:20:17.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.143+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3a40615c0 con 0x7ff38c0a1ac0 2024-03-11T19:20:17.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.173+0000 7ff3a2000640 1 -- 172.21.15.72:0/2213418705 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3a40615c0 con 0x7ff38c0a1ac0 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.175+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 msgr2=0x7ff38c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.175+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 0x7ff38c0a3f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3a419ec10 tx=0x7ff390005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.175+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 msgr2=0x7ff3a419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.175+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff398005ab0 tx=0x7ff398005e50 comp rx=0 tx=0).stop 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 shutdown_connections 2024-03-11T19:20:17.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff38c0a1ac0 0x7ff38c0a3f80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3a4109250 0x7ff3a41a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3a4108890 0x7ff3a419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 --2- 172.21.15.72:0/2213418705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff3a41076a0 0x7ff3a419d700 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 >> 172.21.15.72:0/2213418705 conn(0x7ff3a4076650 msgr2=0x7ff3a410bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 shutdown_connections 2024-03-11T19:20:17.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.176+0000 7ff3aaa00640 1 -- 172.21.15.72:0/2213418705 wait complete. 2024-03-11T19:20:17.197 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-11T19:20:17.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 -- 172.21.15.72:0/608824179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 msgr2=0x7fc7d0104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 --2- 172.21.15.72:0/608824179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0104a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009900 tx=0x7fc7bc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 -- 172.21.15.72:0/608824179 shutdown_connections 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 --2- 172.21.15.72:0/608824179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d0104f50 0x7fc7d010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 --2- 172.21.15.72:0/608824179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0104a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 --2- 172.21.15.72:0/608824179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d01033a0 0x7fc7d01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 -- 172.21.15.72:0/608824179 >> 172.21.15.72:0/608824179 conn(0x7fc7d00fe930 msgr2=0x7fc7d0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 -- 172.21.15.72:0/608824179 shutdown_connections 2024-03-11T19:20:17.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.389+0000 7fc7d8400640 1 -- 172.21.15.72:0/608824179 wait complete. 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.390+0000 7fc7d8400640 1 Processor -- start 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 -- start start 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d01033a0 0x7fc7d0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d0104f50 0x7fc7d01a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d01a0f30 con 0x7fc7d01033a0 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d01a10a0 con 0x7fc7d0104590 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d8400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d01a1260 con 0x7fc7d0104f50 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d5000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d5000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:47110/0 (socket says 172.21.15.72:47110) 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d6400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d0104f50 0x7fc7d01a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d5000640 1 -- 172.21.15.72:0/2957876033 learned_addr learned my addr 172.21.15.72:0/2957876033 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.391+0000 7fc7d5a00640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d01033a0 0x7fc7d0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 -- 172.21.15.72:0/2957876033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d0104f50 msgr2=0x7fc7d01a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d0104f50 0x7fc7d01a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 -- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d01033a0 msgr2=0x7fc7d0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d01033a0 0x7fc7d0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 -- 172.21.15.72:0/2957876033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc009560 con 0x7fc7d0104590 2024-03-11T19:20:17.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d5000640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc02c6a0 tx=0x7fc7bc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:17.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0044e0 con 0x7fc7d0104590 2024-03-11T19:20:17.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7bc004ac0 con 0x7fc7d0104590 2024-03-11T19:20:17.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc03e450 con 0x7fc7d0104590 2024-03-11T19:20:17.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.392+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d01a14e0 con 0x7fc7d0104590 2024-03-11T19:20:17.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.393+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d01a1a40 con 0x7fc7d0104590 2024-03-11T19:20:17.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.394+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d010a6c0 con 0x7fc7d0104590 2024-03-11T19:20:17.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.395+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fc7bc0035f0 con 0x7fc7d0104590 2024-03-11T19:20:17.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.397+0000 7fc7ce000640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 0x7fc7b80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.397+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc7bc0f0350 con 0x7fc7d0104590 2024-03-11T19:20:17.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.397+0000 7fc7d5a00640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 0x7fc7b80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.397+0000 7fc7d5a00640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 0x7fc7b80a4000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc7d01043f0 tx=0x7fc7c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:17.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.399+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7bc0b9270 con 0x7fc7d0104590 2024-03-11T19:20:17.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.627+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7d002ceb0 con 0x7fc7b80a1b40 2024-03-11T19:20:17.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.634+0000 7fc7ce000640 1 -- 172.21.15.72:0/2957876033 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc7d002ceb0 con 0x7fc7b80a1b40 2024-03-11T19:20:17.637 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_1/13cd1ee8-743e-48fd-a290-a198de32115e 2024-03-11T19:20:17.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 msgr2=0x7fc7b80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 0x7fc7b80a4000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc7d01043f0 tx=0x7fc7c400b390 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 msgr2=0x7fc7d0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc02c6a0 tx=0x7fc7bc0042f0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 shutdown_connections 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fc7b80a1b40 0x7fc7b80a4000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d0104f50 0x7fc7d01a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d0104590 0x7fc7d0199980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 --2- 172.21.15.72:0/2957876033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d01033a0 0x7fc7d0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 >> 172.21.15.72:0/2957876033 conn(0x7fc7d00fe930 msgr2=0x7fc7d0107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 shutdown_connections 2024-03-11T19:20:17.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.637+0000 7fc7d8400640 1 -- 172.21.15.72:0/2957876033 wait complete. 2024-03-11T19:20:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-mon[35889]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-mon[35889]: pgmap v343: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-03-11T19:20:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-mon[35889]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-mon[35889]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:20:17.655 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:17 smithi072 ceph-mon[31695]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.656 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:17 smithi072 ceph-mon[31695]: pgmap v343: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-03-11T19:20:17.656 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:17 smithi072 ceph-mon[31695]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.656 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:17 smithi072 ceph-mon[31695]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:20:17.658 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:20:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:17 smithi143 ceph-mon[36013]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:17 smithi143 ceph-mon[36013]: pgmap v343: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 3 op/s 2024-03-11T19:20:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:17 smithi143 ceph-mon[36013]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:17 smithi143 ceph-mon[36013]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 -- 172.21.15.72:0/3177283896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 msgr2=0x7f95bc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 --2- 172.21.15.72:0/3177283896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc108e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009900 tx=0x7f95a802bc30 comp rx=0 tx=0).stop 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 -- 172.21.15.72:0/3177283896 shutdown_connections 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 --2- 172.21.15.72:0/3177283896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc109370 0x7f95bc10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 --2- 172.21.15.72:0/3177283896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc108e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 --2- 172.21.15.72:0/3177283896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc1077c0 0x7f95bc107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 -- 172.21.15.72:0/3177283896 >> 172.21.15.72:0/3177283896 conn(0x7f95bc076790 msgr2=0x7f95bc078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 -- 172.21.15.72:0/3177283896 shutdown_connections 2024-03-11T19:20:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.857+0000 7f95c2800640 1 -- 172.21.15.72:0/3177283896 wait complete. 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.858+0000 7f95c2800640 1 Processor -- start 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.858+0000 7f95c2800640 1 -- start start 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc109370 0x7f95bc1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1a5350 con 0x7f95bc1089b0 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1a54c0 con 0x7f95bc1077c0 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95c2800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1a5680 con 0x7f95bc109370 2024-03-11T19:20:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:47138/0 (socket says 172.21.15.72:47138) 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95baa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 -- 172.21.15.72:0/1354014004 learned_addr learned my addr 172.21.15.72:0/1354014004 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bbe00640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc109370 0x7f95bc1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 -- 172.21.15.72:0/1354014004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc109370 msgr2=0x7f95bc1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc109370 0x7f95bc1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 -- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 msgr2=0x7f95bc19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.859+0000 7f95bb400640 1 -- 172.21.15.72:0/1354014004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009560 con 0x7f95bc1077c0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95bb400640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00e9c0 tx=0x7f95ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0096e0 con 0x7f95bc1077c0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc1a5960 con 0x7f95bc1077c0 2024-03-11T19:20:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95ac00cd80 con 0x7f95bc1077c0 2024-03-11T19:20:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac010430 con 0x7f95bc1077c0 2024-03-11T19:20:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.860+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc1a5df0 con 0x7f95bc1077c0 2024-03-11T19:20:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.861+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc10eae0 con 0x7f95bc1077c0 2024-03-11T19:20:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.863+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f95ac003680 con 0x7f95bc1077c0 2024-03-11T19:20:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.864+0000 7f95b3e00640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 0x7f95a00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.865+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f95ac014070 con 0x7f95bc1077c0 2024-03-11T19:20:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.865+0000 7f95baa00640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 0x7f95a00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.865+0000 7f95baa00640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 0x7f95a00a3f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:17.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:17.869+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95ac0969d0 con 0x7f95bc1077c0 2024-03-11T19:20:18.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.089+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95bc113210 con 0x7f95bc1077c0 2024-03-11T19:20:18.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.090+0000 7f95b3e00640 1 -- 172.21.15.72:0/1354014004 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f95bc113210 con 0x7f95bc1077c0 2024-03-11T19:20:18.094 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:18.094 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:20:18.094 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 msgr2=0x7f95a00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 0x7f95a00a3f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 msgr2=0x7f95bc19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00e9c0 tx=0x7f95ac00ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 shutdown_connections 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f95a00a1ac0 0x7f95a00a3f80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95bc109370 0x7f95bc1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95bc1089b0 0x7f95bc19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 --2- 172.21.15.72:0/1354014004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95bc1077c0 0x7f95bc19d860 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 >> 172.21.15.72:0/1354014004 conn(0x7f95bc076790 msgr2=0x7f95bc10bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 shutdown_connections 2024-03-11T19:20:18.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.093+0000 7f95c2800640 1 -- 172.21.15.72:0/1354014004 wait complete. 2024-03-11T19:20:18.116 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:20:18.145 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: 2024-03-11T19:20:17.686+0000 7fe59fda4700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-11T19:20:18.145 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:20:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: 2024-03-11T19:20:17.686+0000 7fe59fda4700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-11T19:20:18.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 -- 172.21.15.72:0/1757411435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 msgr2=0x7f82c81015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 --2- 172.21.15.72:0/1757411435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c81015c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009860 tx=0x7f82b802bb70 comp rx=0 tx=0).stop 2024-03-11T19:20:18.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 -- 172.21.15.72:0/1757411435 shutdown_connections 2024-03-11T19:20:18.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 --2- 172.21.15.72:0/1757411435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102d50 0x7f82c81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 --2- 172.21.15.72:0/1757411435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102390 0x7f82c8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 --2- 172.21.15.72:0/1757411435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c81015c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.326+0000 7f82cda00640 1 -- 172.21.15.72:0/1757411435 >> 172.21.15.72:0/1757411435 conn(0x7f82c80fc730 msgr2=0x7f82c80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:18.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.327+0000 7f82cda00640 1 -- 172.21.15.72:0/1757411435 shutdown_connections 2024-03-11T19:20:18.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.327+0000 7f82cda00640 1 -- 172.21.15.72:0/1757411435 wait complete. 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.327+0000 7f82cda00640 1 Processor -- start 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 -- start start 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102390 0x7f82c8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819ed20 con 0x7f82c8102d50 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819ee90 con 0x7f82c8102390 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82cda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819f050 con 0x7f82c81011a0 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39860/0 (socket says 172.21.15.72:39860) 2024-03-11T19:20:18.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82c7400640 1 -- 172.21.15.72:0/559960991 learned_addr learned my addr 172.21.15.72:0/559960991 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82c6a00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.328+0000 7f82c6000640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102390 0x7f82c8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 -- 172.21.15.72:0/559960991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 msgr2=0x7f82c8197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c8197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 -- 172.21.15.72:0/559960991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102390 msgr2=0x7f82c8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102390 0x7f82c8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 -- 172.21.15.72:0/559960991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009560 con 0x7f82c8102d50 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82c7400640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82c000eb10 tx=0x7f82c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004b20 con 0x7f82c8102d50 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b8009490 con 0x7f82c8102d50 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82c0004c80 con 0x7f82c8102d50 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0010650 con 0x7f82c8102d50 2024-03-11T19:20:18.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.329+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c819f6b0 con 0x7f82c8102d50 2024-03-11T19:20:18.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.331+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c81084c0 con 0x7f82c8102d50 2024-03-11T19:20:18.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.332+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f82c0005020 con 0x7f82c8102d50 2024-03-11T19:20:18.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.334+0000 7f82bf400640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 0x7f82ac0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.334+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f82c0014070 con 0x7f82c8102d50 2024-03-11T19:20:18.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.334+0000 7f82c6a00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 0x7f82ac0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.335+0000 7f82c6a00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 0x7f82ac0a3f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f82b8034b10 tx=0x7f82b8034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:18.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.338+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82c0096b90 con 0x7f82c8102d50 2024-03-11T19:20:18.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:18 smithi072 ceph-mon[31695]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:18 smithi072 ceph-mon[31695]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:18 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1354014004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:20:18.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.538+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82c802ce40 con 0x7f82c8102d50 2024-03-11T19:20:18.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.540+0000 7f82bf400640 1 -- 172.21.15.72:0/559960991 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24141 (secure 0 0 0) 0x7f82c009a250 con 0x7f82c8102d50 2024-03-11T19:20:18.543 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:18.543 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":90,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:19:47.263294+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.542+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 msgr2=0x7f82ac0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.542+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 0x7f82ac0a3f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f82b8034b10 tx=0x7f82b8034be0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.542+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 msgr2=0x7f82c819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.542+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82c000eb10 tx=0x7f82c000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 shutdown_connections 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82ac0a1ac0 0x7f82ac0a3f80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82c8102d50 0x7f82c819e7e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82c8102390 0x7f82c8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 --2- 172.21.15.72:0/559960991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011a0 0x7f82c8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 >> 172.21.15.72:0/559960991 conn(0x7f82c80fc730 msgr2=0x7f82c8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:18.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 shutdown_connections 2024-03-11T19:20:18.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.543+0000 7f82cda00640 1 -- 172.21.15.72:0/559960991 wait complete. 2024-03-11T19:20:18.565 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-11T19:20:18.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:20:18.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:18 smithi106 ceph-mon[35889]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:18 smithi106 ceph-mon[35889]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:18 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1354014004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:20:18.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:18 smithi143 ceph-mon[36013]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:18 smithi143 ceph-mon[36013]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:18.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:18 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1354014004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 -- 172.21.15.72:0/3485149841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1000104d80 msgr2=0x7f100010b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 --2- 172.21.15.72:0/3485149841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1000104d80 0x7f100010b610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009ca0 tx=0x7f0ff802c100 comp rx=0 tx=0).stop 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 -- 172.21.15.72:0/3485149841 shutdown_connections 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 --2- 172.21.15.72:0/3485149841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1000104d80 0x7f100010b610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 --2- 172.21.15.72:0/3485149841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 0x7f1000104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 --2- 172.21.15.72:0/3485149841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10001031d0 0x7f10001035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 -- 172.21.15.72:0/3485149841 >> 172.21.15.72:0/3485149841 conn(0x7f10000fe760 msgr2=0x7f1000100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:18.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 -- 172.21.15.72:0/3485149841 shutdown_connections 2024-03-11T19:20:18.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.763+0000 7f1005c00640 1 -- 172.21.15.72:0/3485149841 wait complete. 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 Processor -- start 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 -- start start 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 0x7f1000199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1000104d80 0x7f10001a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001a0cf0 con 0x7f10001031d0 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001a0e60 con 0x7f10001043c0 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.764+0000 7f1005c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001a1020 con 0x7f1000104d80 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39878/0 (socket says 172.21.15.72:39878) 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 -- 172.21.15.72:0/2662789233 learned_addr learned my addr 172.21.15.72:0/2662789233 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f1005200640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1000104d80 0x7f10001a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fff400640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 0x7f1000199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 -- 172.21.15.72:0/2662789233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1000104d80 msgr2=0x7f10001a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1000104d80 0x7f10001a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 -- 172.21.15.72:0/2662789233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 msgr2=0x7f1000199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 0x7f1000199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 -- 172.21.15.72:0/2662789233 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8009900 con 0x7f10001031d0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.765+0000 7f0fffe00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e9c0 tx=0x7f0ff000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.766+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00096e0 con 0x7f10001031d0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.766+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ff000cd80 con 0x7f10001031d0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.766+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10001a1300 con 0x7f10001031d0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.766+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0010430 con 0x7f10001031d0 2024-03-11T19:20:18.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.766+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10001a1810 con 0x7f10001031d0 2024-03-11T19:20:18.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.767+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100010a490 con 0x7f10001031d0 2024-03-11T19:20:18.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.768+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0ff0003680 con 0x7f10001031d0 2024-03-11T19:20:18.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.770+0000 7f0ffcc00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 0x7f0fe40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:18.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.770+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0ff0014070 con 0x7f10001031d0 2024-03-11T19:20:18.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.770+0000 7f0fff400640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 0x7f0fe40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:18.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.771+0000 7f0fff400640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 0x7f0fe40a4000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f100019a710 tx=0x7f0fe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:18.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.773+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ff0096970 con 0x7f10001031d0 2024-03-11T19:20:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.972+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f100002ceb0 con 0x7f0fe40a1b40 2024-03-11T19:20:18.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.991+0000 7f0ffcc00640 1 -- 172.21.15.72:0/2662789233 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340650 (secure 0 0 0) 0x7f100002ceb0 con 0x7f0fe40a1b40 2024-03-11T19:20:18.999 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:19.001 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 msgr2=0x7f0fe40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 0x7f0fe40a4000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f100019a710 tx=0x7f0fe8005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 msgr2=0x7f1000199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e9c0 tx=0x7f0ff000ee80 comp rx=0 tx=0).stop 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 shutdown_connections 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fe40a1b40 0x7f0fe40a4000 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1000104d80 0x7f10001a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001043c0 0x7f1000199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 --2- 172.21.15.72:0/2662789233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10001031d0 0x7f1000199200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 >> 172.21.15.72:0/2662789233 conn(0x7f10000fe760 msgr2=0x7f1000107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:19.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.996+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 shutdown_connections 2024-03-11T19:20:19.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:18.997+0000 7f1005c00640 1 -- 172.21.15.72:0/2662789233 wait complete. 2024-03-11T19:20:19.018 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-03-11T19:20:18.066206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633984,"num_objects":58,"num_object_clones":0,"num_object_copies":175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":150,"num_read_kb":167,"num_write":315,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369240,"kb_used_data":11768,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572584,"statfs":{"total":1151940427776,"available":1151562326016,"internally_reserved":0,"allocated":12050432,"data_stored":7595116,"data_compressed":283332,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39117,"num_objects":34,"num_object_clones":0,"num_object_copies":103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":14,"num_read_kb":48,"num_write":86,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006294"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244017+0000","last_change":"2024-03-11T19:19:42.386329+0000","last_active":"2024-03-11T19:19:48.244017+0000","last_peered":"2024-03-11T19:19:48.244017+0000","last_clean":"2024-03-11T19:19:48.244017+0000","last_became_active":"2024-03-11T19:19:42.386041+0000","last_became_peered":"2024-03-11T19:19:42.386041+0000","last_unstale":"2024-03-11T19:19:48.244017+0000","last_undegraded":"2024-03-11T19:19:48.244017+0000","last_fullsized":"2024-03-11T19:19:48.244017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:05:38.258840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246437+0000","last_change":"2024-03-11T19:19:42.381508+0000","last_active":"2024-03-11T19:19:48.246437+0000","last_peered":"2024-03-11T19:19:48.246437+0000","last_clean":"2024-03-11T19:19:48.246437+0000","last_became_active":"2024-03-11T19:19:42.378340+0000","last_became_peered":"2024-03-11T19:19:42.378340+0000","last_unstale":"2024-03-11T19:19:48.246437+0000","last_undegraded":"2024-03-11T19:19:48.246437+0000","last_fullsized":"2024-03-11T19:19:48.246437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:31:06.472488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245720+0000","last_change":"2024-03-11T19:19:42.377904+0000","last_active":"2024-03-11T19:19:48.245720+0000","last_peered":"2024-03-11T19:19:48.245720+0000","last_clean":"2024-03-11T19:19:48.245720+0000","last_became_active":"2024-03-11T19:19:42.376279+0000","last_became_peered":"2024-03-11T19:19:42.376279+0000","last_unstale":"2024-03-11T19:19:48.245720+0000","last_undegraded":"2024-03-11T19:19:48.245720+0000","last_fullsized":"2024-03-11T19:19:48.245720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:52.685543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243316+0000","last_change":"2024-03-11T19:19:42.373261+0000","last_active":"2024-03-11T19:19:48.243316+0000","last_peered":"2024-03-11T19:19:48.243316+0000","last_clean":"2024-03-11T19:19:48.243316+0000","last_became_active":"2024-03-11T19:19:42.372898+0000","last_became_peered":"2024-03-11T19:19:42.372898+0000","last_unstale":"2024-03-11T19:19:48.243316+0000","last_undegraded":"2024-03-11T19:19:48.243316+0000","last_fullsized":"2024-03-11T19:19:48.243316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:22.631405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244507+0000","last_change":"2024-03-11T19:19:42.380979+0000","last_active":"2024-03-11T19:19:48.244507+0000","last_peered":"2024-03-11T19:19:48.244507+0000","last_clean":"2024-03-11T19:19:48.244507+0000","last_became_active":"2024-03-11T19:19:42.380653+0000","last_became_peered":"2024-03-11T19:19:42.380653+0000","last_unstale":"2024-03-11T19:19:48.244507+0000","last_undegraded":"2024-03-11T19:19:48.244507+0000","last_fullsized":"2024-03-11T19:19:48.244507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:39.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243766+0000","last_change":"2024-03-11T19:19:42.378182+0000","last_active":"2024-03-11T19:19:48.243766+0000","last_peered":"2024-03-11T19:19:48.243766+0000","last_clean":"2024-03-11T19:19:48.243766+0000","last_became_active":"2024-03-11T19:19:42.377810+0000","last_became_peered":"2024-03-11T19:19:42.377810+0000","last_unstale":"2024-03-11T19:19:48.243766+0000","last_undegraded":"2024-03-11T19:19:48.243766+0000","last_fullsized":"2024-03-11T19:19:48.243766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:37:20.498630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993271+0000","last_change":"2024-03-11T19:19:42.370872+0000","last_active":"2024-03-11T19:19:47.993271+0000","last_peered":"2024-03-11T19:19:47.993271+0000","last_clean":"2024-03-11T19:19:47.993271+0000","last_became_active":"2024-03-11T19:19:42.370543+0000","last_became_peered":"2024-03-11T19:19:42.370543+0000","last_unstale":"2024-03-11T19:19:47.993271+0000","last_undegraded":"2024-03-11T19:19:47.993271+0000","last_fullsized":"2024-03-11T19:19:47.993271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:48.483035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246873+0000","last_change":"2024-03-11T19:19:42.382124+0000","last_active":"2024-03-11T19:19:48.246873+0000","last_peered":"2024-03-11T19:19:48.246873+0000","last_clean":"2024-03-11T19:19:48.246873+0000","last_became_active":"2024-03-11T19:19:42.381700+0000","last_became_peered":"2024-03-11T19:19:42.381700+0000","last_unstale":"2024-03-11T19:19:48.246873+0000","last_undegraded":"2024-03-11T19:19:48.246873+0000","last_fullsized":"2024-03-11T19:19:48.246873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:21.210191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788672+0000","last_change":"2024-03-11T19:19:42.368501+0000","last_active":"2024-03-11T19:19:47.788672+0000","last_peered":"2024-03-11T19:19:47.788672+0000","last_clean":"2024-03-11T19:19:47.788672+0000","last_became_active":"2024-03-11T19:19:42.368155+0000","last_became_peered":"2024-03-11T19:19:42.368155+0000","last_unstale":"2024-03-11T19:19:47.788672+0000","last_undegraded":"2024-03-11T19:19:47.788672+0000","last_fullsized":"2024-03-11T19:19:47.788672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:55:59.244376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244282+0000","last_change":"2024-03-11T19:19:42.365113+0000","last_active":"2024-03-11T19:19:48.244282+0000","last_peered":"2024-03-11T19:19:48.244282+0000","last_clean":"2024-03-11T19:19:48.244282+0000","last_became_active":"2024-03-11T19:19:42.364710+0000","last_became_peered":"2024-03-11T19:19:42.364710+0000","last_unstale":"2024-03-11T19:19:48.244282+0000","last_undegraded":"2024-03-11T19:19:48.244282+0000","last_fullsized":"2024-03-11T19:19:48.244282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:28.999688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993815+0000","last_change":"2024-03-11T19:19:42.365977+0000","last_active":"2024-03-11T19:19:47.993815+0000","last_peered":"2024-03-11T19:19:47.993815+0000","last_clean":"2024-03-11T19:19:47.993815+0000","last_became_active":"2024-03-11T19:19:42.365220+0000","last_became_peered":"2024-03-11T19:19:42.365220+0000","last_unstale":"2024-03-11T19:19:47.993815+0000","last_undegraded":"2024-03-11T19:19:47.993815+0000","last_fullsized":"2024-03-11T19:19:47.993815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:45.707973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243194+0000","last_change":"2024-03-11T19:19:42.368596+0000","last_active":"2024-03-11T19:19:48.243194+0000","last_peered":"2024-03-11T19:19:48.243194+0000","last_clean":"2024-03-11T19:19:48.243194+0000","last_became_active":"2024-03-11T19:19:42.366526+0000","last_became_peered":"2024-03-11T19:19:42.366526+0000","last_unstale":"2024-03-11T19:19:48.243194+0000","last_undegraded":"2024-03-11T19:19:48.243194+0000","last_fullsized":"2024-03-11T19:19:48.243194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:57.992445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792062+0000","last_change":"2024-03-11T19:19:42.374433+0000","last_active":"2024-03-11T19:19:47.792062+0000","last_peered":"2024-03-11T19:19:47.792062+0000","last_clean":"2024-03-11T19:19:47.792062+0000","last_became_active":"2024-03-11T19:19:42.371924+0000","last_became_peered":"2024-03-11T19:19:42.371924+0000","last_unstale":"2024-03-11T19:19:47.792062+0000","last_undegraded":"2024-03-11T19:19:47.792062+0000","last_fullsized":"2024-03-11T19:19:47.792062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:37.515381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245108+0000","last_change":"2024-03-11T19:19:42.378885+0000","last_active":"2024-03-11T19:19:48.245108+0000","last_peered":"2024-03-11T19:19:48.245108+0000","last_clean":"2024-03-11T19:19:48.245108+0000","last_became_active":"2024-03-11T19:19:42.378505+0000","last_became_peered":"2024-03-11T19:19:42.378505+0000","last_unstale":"2024-03-11T19:19:48.245108+0000","last_undegraded":"2024-03-11T19:19:48.245108+0000","last_fullsized":"2024-03-11T19:19:48.245108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:12.380431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244935+0000","last_change":"2024-03-11T19:19:42.377500+0000","last_active":"2024-03-11T19:19:48.244935+0000","last_peered":"2024-03-11T19:19:48.244935+0000","last_clean":"2024-03-11T19:19:48.244935+0000","last_became_active":"2024-03-11T19:19:42.377083+0000","last_became_peered":"2024-03-11T19:19:42.377083+0000","last_unstale":"2024-03-11T19:19:48.244935+0000","last_undegraded":"2024-03-11T19:19:48.244935+0000","last_fullsized":"2024-03-11T19:19:48.244935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:02:56.146071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788080+0000","last_change":"2024-03-11T19:19:42.375809+0000","last_active":"2024-03-11T19:19:47.788080+0000","last_peered":"2024-03-11T19:19:47.788080+0000","last_clean":"2024-03-11T19:19:47.788080+0000","last_became_active":"2024-03-11T19:19:42.375422+0000","last_became_peered":"2024-03-11T19:19:42.375422+0000","last_unstale":"2024-03-11T19:19:47.788080+0000","last_undegraded":"2024-03-11T19:19:47.788080+0000","last_fullsized":"2024-03-11T19:19:47.788080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:17:21.344587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243732+0000","last_change":"2024-03-11T19:19:42.381932+0000","last_active":"2024-03-11T19:19:48.243732+0000","last_peered":"2024-03-11T19:19:48.243732+0000","last_clean":"2024-03-11T19:19:48.243732+0000","last_became_active":"2024-03-11T19:19:42.381591+0000","last_became_peered":"2024-03-11T19:19:42.381591+0000","last_unstale":"2024-03-11T19:19:48.243732+0000","last_undegraded":"2024-03-11T19:19:48.243732+0000","last_fullsized":"2024-03-11T19:19:48.243732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:03.619050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249551+0000","last_change":"2024-03-11T19:19:42.374924+0000","last_active":"2024-03-11T19:19:48.249551+0000","last_peered":"2024-03-11T19:19:48.249551+0000","last_clean":"2024-03-11T19:19:48.249551+0000","last_became_active":"2024-03-11T19:19:42.374094+0000","last_became_peered":"2024-03-11T19:19:42.374094+0000","last_unstale":"2024-03-11T19:19:48.249551+0000","last_undegraded":"2024-03-11T19:19:48.249551+0000","last_fullsized":"2024-03-11T19:19:48.249551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:15.327665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244606+0000","last_change":"2024-03-11T19:19:42.381807+0000","last_active":"2024-03-11T19:19:48.244606+0000","last_peered":"2024-03-11T19:19:48.244606+0000","last_clean":"2024-03-11T19:19:48.244606+0000","last_became_active":"2024-03-11T19:19:42.381056+0000","last_became_peered":"2024-03-11T19:19:42.381056+0000","last_unstale":"2024-03-11T19:19:48.244606+0000","last_undegraded":"2024-03-11T19:19:48.244606+0000","last_fullsized":"2024-03-11T19:19:48.244606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:00.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243693+0000","last_change":"2024-03-11T19:19:46.825875+0000","last_active":"2024-03-11T19:19:48.243693+0000","last_peered":"2024-03-11T19:19:48.243693+0000","last_clean":"2024-03-11T19:19:48.243693+0000","last_became_active":"2024-03-11T19:19:46.825528+0000","last_became_peered":"2024-03-11T19:19:46.825528+0000","last_unstale":"2024-03-11T19:19:48.243693+0000","last_undegraded":"2024-03-11T19:19:48.243693+0000","last_fullsized":"2024-03-11T19:19:48.243693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:32:38.667775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.240411+0000","last_change":"2024-03-11T19:19:41.393633+0000","last_active":"2024-03-11T19:20:14.240411+0000","last_peered":"2024-03-11T19:20:14.240411+0000","last_clean":"2024-03-11T19:20:14.240411+0000","last_became_active":"2024-03-11T19:19:41.389865+0000","last_became_peered":"2024-03-11T19:19:41.389865+0000","last_unstale":"2024-03-11T19:20:14.240411+0000","last_undegraded":"2024-03-11T19:20:14.240411+0000","last_fullsized":"2024-03-11T19:20:14.240411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:29.251688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249750+0000","last_change":"2024-03-11T19:19:42.392393+0000","last_active":"2024-03-11T19:19:48.249750+0000","last_peered":"2024-03-11T19:19:48.249750+0000","last_clean":"2024-03-11T19:19:48.249750+0000","last_became_active":"2024-03-11T19:19:42.390724+0000","last_became_peered":"2024-03-11T19:19:42.390724+0000","last_unstale":"2024-03-11T19:19:48.249750+0000","last_undegraded":"2024-03-11T19:19:48.249750+0000","last_fullsized":"2024-03-11T19:19:48.249750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:49.410831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.242989+0000","last_change":"2024-03-11T19:19:46.817231+0000","last_active":"2024-03-11T19:19:48.242989+0000","last_peered":"2024-03-11T19:19:48.242989+0000","last_clean":"2024-03-11T19:19:48.242989+0000","last_became_active":"2024-03-11T19:19:46.816818+0000","last_became_peered":"2024-03-11T19:19:46.816818+0000","last_unstale":"2024-03-11T19:19:48.242989+0000","last_undegraded":"2024-03-11T19:19:48.242989+0000","last_fullsized":"2024-03-11T19:19:48.242989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:49.562628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249857+0000","last_change":"2024-03-11T19:19:41.401029+0000","last_active":"2024-03-11T19:19:48.249857+0000","last_peered":"2024-03-11T19:19:48.249857+0000","last_clean":"2024-03-11T19:19:48.249857+0000","last_became_active":"2024-03-11T19:19:41.400659+0000","last_became_peered":"2024-03-11T19:19:41.400659+0000","last_unstale":"2024-03-11T19:19:48.249857+0000","last_undegraded":"2024-03-11T19:19:48.249857+0000","last_fullsized":"2024-03-11T19:19:48.249857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:07.903543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246739+0000","last_change":"2024-03-11T19:19:42.380714+0000","last_active":"2024-03-11T19:19:48.246739+0000","last_peered":"2024-03-11T19:19:48.246739+0000","last_clean":"2024-03-11T19:19:48.246739+0000","last_became_active":"2024-03-11T19:19:42.380064+0000","last_became_peered":"2024-03-11T19:19:42.380064+0000","last_unstale":"2024-03-11T19:19:48.246739+0000","last_undegraded":"2024-03-11T19:19:48.246739+0000","last_fullsized":"2024-03-11T19:19:48.246739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:47.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243622+0000","last_change":"2024-03-11T19:19:46.815643+0000","last_active":"2024-03-11T19:19:48.243622+0000","last_peered":"2024-03-11T19:19:48.243622+0000","last_clean":"2024-03-11T19:19:48.243622+0000","last_became_active":"2024-03-11T19:19:46.815222+0000","last_became_peered":"2024-03-11T19:19:46.815222+0000","last_unstale":"2024-03-11T19:19:48.243622+0000","last_undegraded":"2024-03-11T19:19:48.243622+0000","last_fullsized":"2024-03-11T19:19:48.243622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:44:24.625495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.235525+0000","last_change":"2024-03-11T19:19:41.397810+0000","last_active":"2024-03-11T19:20:14.235525+0000","last_peered":"2024-03-11T19:20:14.235525+0000","last_clean":"2024-03-11T19:20:14.235525+0000","last_became_active":"2024-03-11T19:19:41.396726+0000","last_became_peered":"2024-03-11T19:19:41.396726+0000","last_unstale":"2024-03-11T19:20:14.235525+0000","last_undegraded":"2024-03-11T19:20:14.235525+0000","last_fullsized":"2024-03-11T19:20:14.235525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:26.328409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788336+0000","last_change":"2024-03-11T19:19:42.360595+0000","last_active":"2024-03-11T19:19:47.788336+0000","last_peered":"2024-03-11T19:19:47.788336+0000","last_clean":"2024-03-11T19:19:47.788336+0000","last_became_active":"2024-03-11T19:19:42.360230+0000","last_became_peered":"2024-03-11T19:19:42.360230+0000","last_unstale":"2024-03-11T19:19:47.788336+0000","last_undegraded":"2024-03-11T19:19:47.788336+0000","last_fullsized":"2024-03-11T19:19:47.788336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:45.480391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.242931+0000","last_change":"2024-03-11T19:19:46.817442+0000","last_active":"2024-03-11T19:19:48.242931+0000","last_peered":"2024-03-11T19:19:48.242931+0000","last_clean":"2024-03-11T19:19:48.242931+0000","last_became_active":"2024-03-11T19:19:46.816944+0000","last_became_peered":"2024-03-11T19:19:46.816944+0000","last_unstale":"2024-03-11T19:19:48.242931+0000","last_undegraded":"2024-03-11T19:19:48.242931+0000","last_fullsized":"2024-03-11T19:19:48.242931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:05:43.783450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243550+0000","last_change":"2024-03-11T19:19:41.389608+0000","last_active":"2024-03-11T19:19:48.243550+0000","last_peered":"2024-03-11T19:19:48.243550+0000","last_clean":"2024-03-11T19:19:48.243550+0000","last_became_active":"2024-03-11T19:19:41.389281+0000","last_became_peered":"2024-03-11T19:19:41.389281+0000","last_unstale":"2024-03-11T19:19:48.243550+0000","last_undegraded":"2024-03-11T19:19:48.243550+0000","last_fullsized":"2024-03-11T19:19:48.243550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:46.213494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788893+0000","last_change":"2024-03-11T19:19:42.371444+0000","last_active":"2024-03-11T19:19:47.788893+0000","last_peered":"2024-03-11T19:19:47.788893+0000","last_clean":"2024-03-11T19:19:47.788893+0000","last_became_active":"2024-03-11T19:19:42.371129+0000","last_became_peered":"2024-03-11T19:19:42.371129+0000","last_unstale":"2024-03-11T19:19:47.788893+0000","last_undegraded":"2024-03-11T19:19:47.788893+0000","last_fullsized":"2024-03-11T19:19:47.788893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:04:07.449453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244518+0000","last_change":"2024-03-11T19:19:46.805616+0000","last_active":"2024-03-11T19:19:48.244518+0000","last_peered":"2024-03-11T19:19:48.244518+0000","last_clean":"2024-03-11T19:19:48.244518+0000","last_became_active":"2024-03-11T19:19:46.805069+0000","last_became_peered":"2024-03-11T19:19:46.805069+0000","last_unstale":"2024-03-11T19:19:48.244518+0000","last_undegraded":"2024-03-11T19:19:48.244518+0000","last_fullsized":"2024-03-11T19:19:48.244518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:47:23.197283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232608+0000","last_change":"2024-03-11T19:19:41.392375+0000","last_active":"2024-03-11T19:20:13.232608+0000","last_peered":"2024-03-11T19:20:13.232608+0000","last_clean":"2024-03-11T19:20:13.232608+0000","last_became_active":"2024-03-11T19:19:41.392057+0000","last_became_peered":"2024-03-11T19:19:41.392057+0000","last_unstale":"2024-03-11T19:20:13.232608+0000","last_undegraded":"2024-03-11T19:20:13.232608+0000","last_fullsized":"2024-03-11T19:20:13.232608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:12.842671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244284+0000","last_change":"2024-03-11T19:19:42.373255+0000","last_active":"2024-03-11T19:19:48.244284+0000","last_peered":"2024-03-11T19:19:48.244284+0000","last_clean":"2024-03-11T19:19:48.244284+0000","last_became_active":"2024-03-11T19:19:42.372900+0000","last_became_peered":"2024-03-11T19:19:42.372900+0000","last_unstale":"2024-03-11T19:19:48.244284+0000","last_undegraded":"2024-03-11T19:19:48.244284+0000","last_fullsized":"2024-03-11T19:19:48.244284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:56:45.517189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244494+0000","last_change":"2024-03-11T19:19:46.819292+0000","last_active":"2024-03-11T19:19:48.244494+0000","last_peered":"2024-03-11T19:19:48.244494+0000","last_clean":"2024-03-11T19:19:48.244494+0000","last_became_active":"2024-03-11T19:19:46.818932+0000","last_became_peered":"2024-03-11T19:19:46.818932+0000","last_unstale":"2024-03-11T19:19:48.244494+0000","last_undegraded":"2024-03-11T19:19:48.244494+0000","last_fullsized":"2024-03-11T19:19:48.244494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:59.568422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.234747+0000","last_change":"2024-03-11T19:19:41.385591+0000","last_active":"2024-03-11T19:20:14.234747+0000","last_peered":"2024-03-11T19:20:14.234747+0000","last_clean":"2024-03-11T19:20:14.234747+0000","last_became_active":"2024-03-11T19:19:41.385236+0000","last_became_peered":"2024-03-11T19:19:41.385236+0000","last_unstale":"2024-03-11T19:20:14.234747+0000","last_undegraded":"2024-03-11T19:20:14.234747+0000","last_fullsized":"2024-03-11T19:20:14.234747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:55:55.558948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788683+0000","last_change":"2024-03-11T19:19:42.380934+0000","last_active":"2024-03-11T19:19:47.788683+0000","last_peered":"2024-03-11T19:19:47.788683+0000","last_clean":"2024-03-11T19:19:47.788683+0000","last_became_active":"2024-03-11T19:19:42.380602+0000","last_became_peered":"2024-03-11T19:19:42.380602+0000","last_unstale":"2024-03-11T19:19:47.788683+0000","last_undegraded":"2024-03-11T19:19:47.788683+0000","last_fullsized":"2024-03-11T19:19:47.788683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:58:26.813552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243663+0000","last_change":"2024-03-11T19:19:46.814227+0000","last_active":"2024-03-11T19:19:48.243663+0000","last_peered":"2024-03-11T19:19:48.243663+0000","last_clean":"2024-03-11T19:19:48.243663+0000","last_became_active":"2024-03-11T19:19:46.813874+0000","last_became_peered":"2024-03-11T19:19:46.813874+0000","last_unstale":"2024-03-11T19:19:48.243663+0000","last_undegraded":"2024-03-11T19:19:48.243663+0000","last_fullsized":"2024-03-11T19:19:48.243663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:21.493313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.242599+0000","last_change":"2024-03-11T19:19:41.392245+0000","last_active":"2024-03-11T19:20:14.242599+0000","last_peered":"2024-03-11T19:20:14.242599+0000","last_clean":"2024-03-11T19:20:14.242599+0000","last_became_active":"2024-03-11T19:19:41.391928+0000","last_became_peered":"2024-03-11T19:19:41.391928+0000","last_unstale":"2024-03-11T19:20:14.242599+0000","last_undegraded":"2024-03-11T19:20:14.242599+0000","last_fullsized":"2024-03-11T19:20:14.242599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:20.556826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246600+0000","last_change":"2024-03-11T19:19:42.389431+0000","last_active":"2024-03-11T19:19:48.246600+0000","last_peered":"2024-03-11T19:19:48.246600+0000","last_clean":"2024-03-11T19:19:48.246600+0000","last_became_active":"2024-03-11T19:19:42.387885+0000","last_became_peered":"2024-03-11T19:19:42.387885+0000","last_unstale":"2024-03-11T19:19:48.246600+0000","last_undegraded":"2024-03-11T19:19:48.246600+0000","last_fullsized":"2024-03-11T19:19:48.246600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:43.753079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245978+0000","last_change":"2024-03-11T19:19:46.823474+0000","last_active":"2024-03-11T19:19:48.245978+0000","last_peered":"2024-03-11T19:19:48.245978+0000","last_clean":"2024-03-11T19:19:48.245978+0000","last_became_active":"2024-03-11T19:19:46.823008+0000","last_became_peered":"2024-03-11T19:19:46.823008+0000","last_unstale":"2024-03-11T19:19:48.245978+0000","last_undegraded":"2024-03-11T19:19:48.245978+0000","last_fullsized":"2024-03-11T19:19:48.245978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:28.381809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788180+0000","last_change":"2024-03-11T19:19:41.395332+0000","last_active":"2024-03-11T19:19:47.788180+0000","last_peered":"2024-03-11T19:19:47.788180+0000","last_clean":"2024-03-11T19:19:47.788180+0000","last_became_active":"2024-03-11T19:19:41.394989+0000","last_became_peered":"2024-03-11T19:19:41.394989+0000","last_unstale":"2024-03-11T19:19:47.788180+0000","last_undegraded":"2024-03-11T19:19:47.788180+0000","last_fullsized":"2024-03-11T19:19:47.788180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:57:31.486891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.247071+0000","last_change":"2024-03-11T19:19:42.393223+0000","last_active":"2024-03-11T19:19:48.247071+0000","last_peered":"2024-03-11T19:19:48.247071+0000","last_clean":"2024-03-11T19:19:48.247071+0000","last_became_active":"2024-03-11T19:19:42.392787+0000","last_became_peered":"2024-03-11T19:19:42.392787+0000","last_unstale":"2024-03-11T19:19:48.247071+0000","last_undegraded":"2024-03-11T19:19:48.247071+0000","last_fullsized":"2024-03-11T19:19:48.247071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:32:50.693896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246192+0000","last_change":"2024-03-11T19:19:46.808305+0000","last_active":"2024-03-11T19:19:48.246192+0000","last_peered":"2024-03-11T19:19:48.246192+0000","last_clean":"2024-03-11T19:19:48.246192+0000","last_became_active":"2024-03-11T19:19:46.807816+0000","last_became_peered":"2024-03-11T19:19:46.807816+0000","last_unstale":"2024-03-11T19:19:48.246192+0000","last_undegraded":"2024-03-11T19:19:48.246192+0000","last_fullsized":"2024-03-11T19:19:48.246192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:55:43.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:16.245590+0000","last_change":"2024-03-11T19:19:41.389335+0000","last_active":"2024-03-11T19:20:16.245590+0000","last_peered":"2024-03-11T19:20:16.245590+0000","last_clean":"2024-03-11T19:20:16.245590+0000","last_became_active":"2024-03-11T19:19:41.389023+0000","last_became_peered":"2024-03-11T19:19:41.389023+0000","last_unstale":"2024-03-11T19:20:16.245590+0000","last_undegraded":"2024-03-11T19:20:16.245590+0000","last_fullsized":"2024-03-11T19:20:16.245590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:12:59.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788567+0000","last_change":"2024-03-11T19:19:42.378379+0000","last_active":"2024-03-11T19:19:47.788567+0000","last_peered":"2024-03-11T19:19:47.788567+0000","last_clean":"2024-03-11T19:19:47.788567+0000","last_became_active":"2024-03-11T19:19:42.378033+0000","last_became_peered":"2024-03-11T19:19:42.378033+0000","last_unstale":"2024-03-11T19:19:47.788567+0000","last_undegraded":"2024-03-11T19:19:47.788567+0000","last_fullsized":"2024-03-11T19:19:47.788567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:24.154420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249448+0000","last_change":"2024-03-11T19:19:46.809464+0000","last_active":"2024-03-11T19:19:48.249448+0000","last_peered":"2024-03-11T19:19:48.249448+0000","last_clean":"2024-03-11T19:19:48.249448+0000","last_became_active":"2024-03-11T19:19:46.808955+0000","last_became_peered":"2024-03-11T19:19:46.808955+0000","last_unstale":"2024-03-11T19:19:48.249448+0000","last_undegraded":"2024-03-11T19:19:48.249448+0000","last_fullsized":"2024-03-11T19:19:48.249448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:42.088992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232018+0000","last_change":"2024-03-11T19:19:41.395619+0000","last_active":"2024-03-11T19:20:13.232018+0000","last_peered":"2024-03-11T19:20:13.232018+0000","last_clean":"2024-03-11T19:20:13.232018+0000","last_became_active":"2024-03-11T19:19:41.394658+0000","last_became_peered":"2024-03-11T19:19:41.394658+0000","last_unstale":"2024-03-11T19:20:13.232018+0000","last_undegraded":"2024-03-11T19:20:13.232018+0000","last_fullsized":"2024-03-11T19:20:13.232018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:10:39.020508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.247467+0000","last_change":"2024-03-11T19:19:42.380416+0000","last_active":"2024-03-11T19:19:48.247467+0000","last_peered":"2024-03-11T19:19:48.247467+0000","last_clean":"2024-03-11T19:19:48.247467+0000","last_became_active":"2024-03-11T19:19:42.379625+0000","last_became_peered":"2024-03-11T19:19:42.379625+0000","last_unstale":"2024-03-11T19:19:48.247467+0000","last_undegraded":"2024-03-11T19:19:48.247467+0000","last_fullsized":"2024-03-11T19:19:48.247467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:45:45.374211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243508+0000","last_change":"2024-03-11T19:19:46.808005+0000","last_active":"2024-03-11T19:19:48.243508+0000","last_peered":"2024-03-11T19:19:48.243508+0000","last_clean":"2024-03-11T19:19:48.243508+0000","last_became_active":"2024-03-11T19:19:46.807549+0000","last_became_peered":"2024-03-11T19:19:46.807549+0000","last_unstale":"2024-03-11T19:19:48.243508+0000","last_undegraded":"2024-03-11T19:19:48.243508+0000","last_fullsized":"2024-03-11T19:19:48.243508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:17.171846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789063+0000","last_change":"2024-03-11T19:19:41.398755+0000","last_active":"2024-03-11T19:19:47.789063+0000","last_peered":"2024-03-11T19:19:47.789063+0000","last_clean":"2024-03-11T19:19:47.789063+0000","last_became_active":"2024-03-11T19:19:41.398406+0000","last_became_peered":"2024-03-11T19:19:41.398406+0000","last_unstale":"2024-03-11T19:19:47.789063+0000","last_undegraded":"2024-03-11T19:19:47.789063+0000","last_fullsized":"2024-03-11T19:19:47.789063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:37.793328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788502+0000","last_change":"2024-03-11T19:19:42.364757+0000","last_active":"2024-03-11T19:19:47.788502+0000","last_peered":"2024-03-11T19:19:47.788502+0000","last_clean":"2024-03-11T19:19:47.788502+0000","last_became_active":"2024-03-11T19:19:42.364432+0000","last_became_peered":"2024-03-11T19:19:42.364432+0000","last_unstale":"2024-03-11T19:19:47.788502+0000","last_undegraded":"2024-03-11T19:19:47.788502+0000","last_fullsized":"2024-03-11T19:19:47.788502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:29.226200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246975+0000","last_change":"2024-03-11T19:19:46.809300+0000","last_active":"2024-03-11T19:19:48.246975+0000","last_peered":"2024-03-11T19:19:48.246975+0000","last_clean":"2024-03-11T19:19:48.246975+0000","last_became_active":"2024-03-11T19:19:46.807327+0000","last_became_peered":"2024-03-11T19:19:46.807327+0000","last_unstale":"2024-03-11T19:19:48.246975+0000","last_undegraded":"2024-03-11T19:19:48.246975+0000","last_fullsized":"2024-03-11T19:19:48.246975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:37:54.082436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'25","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:17.250107+0000","last_change":"2024-03-11T19:19:41.392270+0000","last_active":"2024-03-11T19:20:17.250107+0000","last_peered":"2024-03-11T19:20:17.250107+0000","last_clean":"2024-03-11T19:20:17.250107+0000","last_became_active":"2024-03-11T19:19:41.391559+0000","last_became_peered":"2024-03-11T19:19:41.391559+0000","last_unstale":"2024-03-11T19:20:17.250107+0000","last_undegraded":"2024-03-11T19:20:17.250107+0000","last_fullsized":"2024-03-11T19:20:17.250107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:45.655610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":44,"num_write":26,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244864+0000","last_change":"2024-03-11T19:19:42.368323+0000","last_active":"2024-03-11T19:19:48.244864+0000","last_peered":"2024-03-11T19:19:48.244864+0000","last_clean":"2024-03-11T19:19:48.244864+0000","last_became_active":"2024-03-11T19:19:42.367973+0000","last_became_peered":"2024-03-11T19:19:42.367973+0000","last_unstale":"2024-03-11T19:19:48.244864+0000","last_undegraded":"2024-03-11T19:19:48.244864+0000","last_fullsized":"2024-03-11T19:19:48.244864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:11.573348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789347+0000","last_change":"2024-03-11T19:19:46.822255+0000","last_active":"2024-03-11T19:19:47.789347+0000","last_peered":"2024-03-11T19:19:47.789347+0000","last_clean":"2024-03-11T19:19:47.789347+0000","last_became_active":"2024-03-11T19:19:46.821306+0000","last_became_peered":"2024-03-11T19:19:46.821306+0000","last_unstale":"2024-03-11T19:19:47.789347+0000","last_undegraded":"2024-03-11T19:19:47.789347+0000","last_fullsized":"2024-03-11T19:19:47.789347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:22.353676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232403+0000","last_change":"2024-03-11T19:19:41.397723+0000","last_active":"2024-03-11T19:20:13.232403+0000","last_peered":"2024-03-11T19:20:13.232403+0000","last_clean":"2024-03-11T19:20:13.232403+0000","last_became_active":"2024-03-11T19:19:41.397394+0000","last_became_peered":"2024-03-11T19:19:41.397394+0000","last_unstale":"2024-03-11T19:20:13.232403+0000","last_undegraded":"2024-03-11T19:20:13.232403+0000","last_fullsized":"2024-03-11T19:20:13.232403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:48:42.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792126+0000","last_change":"2024-03-11T19:19:42.389254+0000","last_active":"2024-03-11T19:19:47.792126+0000","last_peered":"2024-03-11T19:19:47.792126+0000","last_clean":"2024-03-11T19:19:47.792126+0000","last_became_active":"2024-03-11T19:19:42.388861+0000","last_became_peered":"2024-03-11T19:19:42.388861+0000","last_unstale":"2024-03-11T19:19:47.792126+0000","last_undegraded":"2024-03-11T19:19:47.792126+0000","last_fullsized":"2024-03-11T19:19:47.792126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:03.308802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243947+0000","last_change":"2024-03-11T19:19:46.821667+0000","last_active":"2024-03-11T19:19:48.243947+0000","last_peered":"2024-03-11T19:19:48.243947+0000","last_clean":"2024-03-11T19:19:48.243947+0000","last_became_active":"2024-03-11T19:19:46.821238+0000","last_became_peered":"2024-03-11T19:19:46.821238+0000","last_unstale":"2024-03-11T19:19:48.243947+0000","last_undegraded":"2024-03-11T19:19:48.243947+0000","last_fullsized":"2024-03-11T19:19:48.243947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:30.997248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.233217+0000","last_change":"2024-03-11T19:19:41.403277+0000","last_active":"2024-03-11T19:20:13.233217+0000","last_peered":"2024-03-11T19:20:13.233217+0000","last_clean":"2024-03-11T19:20:13.233217+0000","last_became_active":"2024-03-11T19:19:41.402875+0000","last_became_peered":"2024-03-11T19:19:41.402875+0000","last_unstale":"2024-03-11T19:20:13.233217+0000","last_undegraded":"2024-03-11T19:20:13.233217+0000","last_fullsized":"2024-03-11T19:20:13.233217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.937257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993609+0000","last_change":"2024-03-11T19:19:42.376244+0000","last_active":"2024-03-11T19:19:47.993609+0000","last_peered":"2024-03-11T19:19:47.993609+0000","last_clean":"2024-03-11T19:19:47.993609+0000","last_became_active":"2024-03-11T19:19:42.375896+0000","last_became_peered":"2024-03-11T19:19:42.375896+0000","last_unstale":"2024-03-11T19:19:47.993609+0000","last_undegraded":"2024-03-11T19:19:47.993609+0000","last_fullsized":"2024-03-11T19:19:47.993609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:46.295898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249767+0000","last_change":"2024-03-11T19:19:46.811977+0000","last_active":"2024-03-11T19:19:48.249767+0000","last_peered":"2024-03-11T19:19:48.249767+0000","last_clean":"2024-03-11T19:19:48.249767+0000","last_became_active":"2024-03-11T19:19:46.811570+0000","last_became_peered":"2024-03-11T19:19:46.811570+0000","last_unstale":"2024-03-11T19:19:48.249767+0000","last_undegraded":"2024-03-11T19:19:48.249767+0000","last_fullsized":"2024-03-11T19:19:48.249767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:00:18.329040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.235103+0000","last_change":"2024-03-11T19:19:41.387779+0000","last_active":"2024-03-11T19:20:14.235103+0000","last_peered":"2024-03-11T19:20:14.235103+0000","last_clean":"2024-03-11T19:20:14.235103+0000","last_became_active":"2024-03-11T19:19:41.387424+0000","last_became_peered":"2024-03-11T19:19:41.387424+0000","last_unstale":"2024-03-11T19:20:14.235103+0000","last_undegraded":"2024-03-11T19:20:14.235103+0000","last_fullsized":"2024-03-11T19:20:14.235103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:49.936069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993327+0000","last_change":"2024-03-11T19:19:42.367664+0000","last_active":"2024-03-11T19:19:47.993327+0000","last_peered":"2024-03-11T19:19:47.993327+0000","last_clean":"2024-03-11T19:19:47.993327+0000","last_became_active":"2024-03-11T19:19:42.367343+0000","last_became_peered":"2024-03-11T19:19:42.367343+0000","last_unstale":"2024-03-11T19:19:47.993327+0000","last_undegraded":"2024-03-11T19:19:47.993327+0000","last_fullsized":"2024-03-11T19:19:47.993327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:29.215094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792428+0000","last_change":"2024-03-11T19:19:46.821832+0000","last_active":"2024-03-11T19:19:47.792428+0000","last_peered":"2024-03-11T19:19:47.792428+0000","last_clean":"2024-03-11T19:19:47.792428+0000","last_became_active":"2024-03-11T19:19:46.821183+0000","last_became_peered":"2024-03-11T19:19:46.821183+0000","last_unstale":"2024-03-11T19:19:47.792428+0000","last_undegraded":"2024-03-11T19:19:47.792428+0000","last_fullsized":"2024-03-11T19:19:47.792428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:56.293370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.240574+0000","last_change":"2024-03-11T19:19:41.400790+0000","last_active":"2024-03-11T19:20:14.240574+0000","last_peered":"2024-03-11T19:20:14.240574+0000","last_clean":"2024-03-11T19:20:14.240574+0000","last_became_active":"2024-03-11T19:19:41.400456+0000","last_became_peered":"2024-03-11T19:19:41.400456+0000","last_unstale":"2024-03-11T19:20:14.240574+0000","last_undegraded":"2024-03-11T19:20:14.240574+0000","last_fullsized":"2024-03-11T19:20:14.240574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:00:18.195113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788315+0000","last_change":"2024-03-11T19:19:42.370608+0000","last_active":"2024-03-11T19:19:47.788315+0000","last_peered":"2024-03-11T19:19:47.788315+0000","last_clean":"2024-03-11T19:19:47.788315+0000","last_became_active":"2024-03-11T19:19:42.370259+0000","last_became_peered":"2024-03-11T19:19:42.370259+0000","last_unstale":"2024-03-11T19:19:47.788315+0000","last_undegraded":"2024-03-11T19:19:47.788315+0000","last_fullsized":"2024-03-11T19:19:47.788315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:50:05.778074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788431+0000","last_change":"2024-03-11T19:19:46.826437+0000","last_active":"2024-03-11T19:19:47.788431+0000","last_peered":"2024-03-11T19:19:47.788431+0000","last_clean":"2024-03-11T19:19:47.788431+0000","last_became_active":"2024-03-11T19:19:46.825964+0000","last_became_peered":"2024-03-11T19:19:46.825964+0000","last_unstale":"2024-03-11T19:19:47.788431+0000","last_undegraded":"2024-03-11T19:19:47.788431+0000","last_fullsized":"2024-03-11T19:19:47.788431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:07.970696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244196+0000","last_change":"2024-03-11T19:19:41.388747+0000","last_active":"2024-03-11T19:19:48.244196+0000","last_peered":"2024-03-11T19:19:48.244196+0000","last_clean":"2024-03-11T19:19:48.244196+0000","last_became_active":"2024-03-11T19:19:41.388408+0000","last_became_peered":"2024-03-11T19:19:41.388408+0000","last_unstale":"2024-03-11T19:19:48.244196+0000","last_undegraded":"2024-03-11T19:19:48.244196+0000","last_fullsized":"2024-03-11T19:19:48.244196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:32:20.195039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244109+0000","last_change":"2024-03-11T19:19:42.371432+0000","last_active":"2024-03-11T19:19:48.244109+0000","last_peered":"2024-03-11T19:19:48.244109+0000","last_clean":"2024-03-11T19:19:48.244109+0000","last_became_active":"2024-03-11T19:19:42.371084+0000","last_became_peered":"2024-03-11T19:19:42.371084+0000","last_unstale":"2024-03-11T19:19:48.244109+0000","last_undegraded":"2024-03-11T19:19:48.244109+0000","last_fullsized":"2024-03-11T19:19:48.244109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:10:50.386891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244395+0000","last_change":"2024-03-11T19:19:46.934632+0000","last_active":"2024-03-11T19:19:48.244395+0000","last_peered":"2024-03-11T19:19:48.244395+0000","last_clean":"2024-03-11T19:19:48.244395+0000","last_became_active":"2024-03-11T19:19:46.934172+0000","last_became_peered":"2024-03-11T19:19:46.934172+0000","last_unstale":"2024-03-11T19:19:48.244395+0000","last_undegraded":"2024-03-11T19:19:48.244395+0000","last_fullsized":"2024-03-11T19:19:48.244395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:01:21.675154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249602+0000","last_change":"2024-03-11T19:19:41.397599+0000","last_active":"2024-03-11T19:19:48.249602+0000","last_peered":"2024-03-11T19:19:48.249602+0000","last_clean":"2024-03-11T19:19:48.249602+0000","last_became_active":"2024-03-11T19:19:41.397206+0000","last_became_peered":"2024-03-11T19:19:41.397206+0000","last_unstale":"2024-03-11T19:19:48.249602+0000","last_undegraded":"2024-03-11T19:19:48.249602+0000","last_fullsized":"2024-03-11T19:19:48.249602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:24:45.835170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244126+0000","last_change":"2024-03-11T19:19:42.387716+0000","last_active":"2024-03-11T19:19:48.244126+0000","last_peered":"2024-03-11T19:19:48.244126+0000","last_clean":"2024-03-11T19:19:48.244126+0000","last_became_active":"2024-03-11T19:19:42.387370+0000","last_became_peered":"2024-03-11T19:19:42.387370+0000","last_unstale":"2024-03-11T19:19:48.244126+0000","last_undegraded":"2024-03-11T19:19:48.244126+0000","last_fullsized":"2024-03-11T19:19:48.244126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:50:22.852284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792546+0000","last_change":"2024-03-11T19:19:46.827980+0000","last_active":"2024-03-11T19:19:47.792546+0000","last_peered":"2024-03-11T19:19:47.792546+0000","last_clean":"2024-03-11T19:19:47.792546+0000","last_became_active":"2024-03-11T19:19:46.827317+0000","last_became_peered":"2024-03-11T19:19:46.827317+0000","last_unstale":"2024-03-11T19:19:47.792546+0000","last_undegraded":"2024-03-11T19:19:47.792546+0000","last_fullsized":"2024-03-11T19:19:47.792546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:55:15.266791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993451+0000","last_change":"2024-03-11T19:19:41.394714+0000","last_active":"2024-03-11T19:19:47.993451+0000","last_peered":"2024-03-11T19:19:47.993451+0000","last_clean":"2024-03-11T19:19:47.993451+0000","last_became_active":"2024-03-11T19:19:41.394404+0000","last_became_peered":"2024-03-11T19:19:41.394404+0000","last_unstale":"2024-03-11T19:19:47.993451+0000","last_undegraded":"2024-03-11T19:19:47.993451+0000","last_fullsized":"2024-03-11T19:19:47.993451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:15.960037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243905+0000","last_change":"2024-03-11T19:19:42.381062+0000","last_active":"2024-03-11T19:19:48.243905+0000","last_peered":"2024-03-11T19:19:48.243905+0000","last_clean":"2024-03-11T19:19:48.243905+0000","last_became_active":"2024-03-11T19:19:42.380744+0000","last_became_peered":"2024-03-11T19:19:42.380744+0000","last_unstale":"2024-03-11T19:19:48.243905+0000","last_undegraded":"2024-03-11T19:19:48.243905+0000","last_fullsized":"2024-03-11T19:19:48.243905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:08.767877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792802+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:19:47.792802+0000","last_peered":"2024-03-11T19:19:47.792802+0000","last_clean":"2024-03-11T19:19:47.792802+0000","last_became_active":"2024-03-11T19:19:46.934195+0000","last_became_peered":"2024-03-11T19:19:46.934195+0000","last_unstale":"2024-03-11T19:19:47.792802+0000","last_undegraded":"2024-03-11T19:19:47.792802+0000","last_fullsized":"2024-03-11T19:19:47.792802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:33:19.268162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788267+0000","last_change":"2024-03-11T19:19:42.366940+0000","last_active":"2024-03-11T19:19:47.788267+0000","last_peered":"2024-03-11T19:19:47.788267+0000","last_clean":"2024-03-11T19:19:47.788267+0000","last_became_active":"2024-03-11T19:19:42.366628+0000","last_became_peered":"2024-03-11T19:19:42.366628+0000","last_unstale":"2024-03-11T19:19:47.788267+0000","last_undegraded":"2024-03-11T19:19:47.788267+0000","last_fullsized":"2024-03-11T19:19:47.788267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:05:37.175276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.243387+0000","last_change":"2024-03-11T19:19:41.389919+0000","last_active":"2024-03-11T19:20:14.243387+0000","last_peered":"2024-03-11T19:20:14.243387+0000","last_clean":"2024-03-11T19:20:14.243387+0000","last_became_active":"2024-03-11T19:19:41.389471+0000","last_became_peered":"2024-03-11T19:19:41.389471+0000","last_unstale":"2024-03-11T19:20:14.243387+0000","last_undegraded":"2024-03-11T19:20:14.243387+0000","last_fullsized":"2024-03-11T19:20:14.243387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:41.068642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993767+0000","last_change":"2024-03-11T19:19:46.804222+0000","last_active":"2024-03-11T19:19:47.993767+0000","last_peered":"2024-03-11T19:19:47.993767+0000","last_clean":"2024-03-11T19:19:47.993767+0000","last_became_active":"2024-03-11T19:19:46.803756+0000","last_became_peered":"2024-03-11T19:19:46.803756+0000","last_unstale":"2024-03-11T19:19:47.993767+0000","last_undegraded":"2024-03-11T19:19:47.993767+0000","last_fullsized":"2024-03-11T19:19:47.993767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:39.170492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248087+0000","last_change":"2024-03-11T19:19:42.380995+0000","last_active":"2024-03-11T19:19:48.248087+0000","last_peered":"2024-03-11T19:19:48.248087+0000","last_clean":"2024-03-11T19:19:48.248087+0000","last_became_active":"2024-03-11T19:19:42.380672+0000","last_became_peered":"2024-03-11T19:19:42.380672+0000","last_unstale":"2024-03-11T19:19:48.248087+0000","last_undegraded":"2024-03-11T19:19:48.248087+0000","last_fullsized":"2024-03-11T19:19:48.248087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:24:01.289629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.234104+0000","last_change":"2024-03-11T19:19:41.391501+0000","last_active":"2024-03-11T19:20:13.234104+0000","last_peered":"2024-03-11T19:20:13.234104+0000","last_clean":"2024-03-11T19:20:13.234104+0000","last_became_active":"2024-03-11T19:19:41.391148+0000","last_became_peered":"2024-03-11T19:19:41.391148+0000","last_unstale":"2024-03-11T19:20:13.234104+0000","last_undegraded":"2024-03-11T19:20:13.234104+0000","last_fullsized":"2024-03-11T19:20:13.234104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:46:18.957547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993162+0000","last_change":"2024-03-11T19:18:21.278121+0000","last_active":"2024-03-11T19:19:47.993162+0000","last_peered":"2024-03-11T19:19:47.993162+0000","last_clean":"2024-03-11T19:19:47.993162+0000","last_became_active":"2024-03-11T19:18:21.277678+0000","last_became_peered":"2024-03-11T19:18:21.277678+0000","last_unstale":"2024-03-11T19:19:47.993162+0000","last_undegraded":"2024-03-11T19:19:47.993162+0000","last_fullsized":"2024-03-11T19:19:47.993162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_clean_scrub_stamp":"2024-03-11T19:14:20.582658+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:19.162992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788982+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:19:47.788982+0000","last_peered":"2024-03-11T19:19:47.788982+0000","last_clean":"2024-03-11T19:19:47.788982+0000","last_became_active":"2024-03-11T19:19:46.934304+0000","last_became_peered":"2024-03-11T19:19:46.934304+0000","last_unstale":"2024-03-11T19:19:47.788982+0000","last_undegraded":"2024-03-11T19:19:47.788982+0000","last_fullsized":"2024-03-11T19:19:47.788982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:34:47.392139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.791525+0000","last_change":"2024-03-11T19:19:42.363283+0000","last_active":"2024-03-11T19:19:47.791525+0000","last_peered":"2024-03-11T19:19:47.791525+0000","last_clean":"2024-03-11T19:19:47.791525+0000","last_became_active":"2024-03-11T19:19:42.362930+0000","last_became_peered":"2024-03-11T19:19:42.362930+0000","last_unstale":"2024-03-11T19:19:47.791525+0000","last_undegraded":"2024-03-11T19:19:47.791525+0000","last_fullsized":"2024-03-11T19:19:47.791525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:16:36.141585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.234905+0000","last_change":"2024-03-11T19:19:41.391776+0000","last_active":"2024-03-11T19:20:14.234905+0000","last_peered":"2024-03-11T19:20:14.234905+0000","last_clean":"2024-03-11T19:20:14.234905+0000","last_became_active":"2024-03-11T19:19:41.391459+0000","last_became_peered":"2024-03-11T19:19:41.391459+0000","last_unstale":"2024-03-11T19:20:14.234905+0000","last_undegraded":"2024-03-11T19:20:14.234905+0000","last_fullsized":"2024-03-11T19:20:14.234905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:36.457307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243595+0000","last_change":"2024-03-11T19:19:46.821662+0000","last_active":"2024-03-11T19:19:48.243595+0000","last_peered":"2024-03-11T19:19:48.243595+0000","last_clean":"2024-03-11T19:19:48.243595+0000","last_became_active":"2024-03-11T19:19:46.821199+0000","last_became_peered":"2024-03-11T19:19:46.821199+0000","last_unstale":"2024-03-11T19:19:48.243595+0000","last_undegraded":"2024-03-11T19:19:48.243595+0000","last_fullsized":"2024-03-11T19:19:48.243595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:08:39.658398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244966+0000","last_change":"2024-03-11T19:19:41.399338+0000","last_active":"2024-03-11T19:19:48.244966+0000","last_peered":"2024-03-11T19:19:48.244966+0000","last_clean":"2024-03-11T19:19:48.244966+0000","last_became_active":"2024-03-11T19:19:41.399027+0000","last_became_peered":"2024-03-11T19:19:41.399027+0000","last_unstale":"2024-03-11T19:19:48.244966+0000","last_undegraded":"2024-03-11T19:19:48.244966+0000","last_fullsized":"2024-03-11T19:19:48.244966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:26:39.157631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792816+0000","last_change":"2024-03-11T19:19:42.380304+0000","last_active":"2024-03-11T19:19:47.792816+0000","last_peered":"2024-03-11T19:19:47.792816+0000","last_clean":"2024-03-11T19:19:47.792816+0000","last_became_active":"2024-03-11T19:19:42.378768+0000","last_became_peered":"2024-03-11T19:19:42.378768+0000","last_unstale":"2024-03-11T19:19:47.792816+0000","last_undegraded":"2024-03-11T19:19:47.792816+0000","last_fullsized":"2024-03-11T19:19:47.792816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:31.465876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246325+0000","last_change":"2024-03-11T19:19:46.934262+0000","last_active":"2024-03-11T19:19:48.246325+0000","last_peered":"2024-03-11T19:19:48.246325+0000","last_clean":"2024-03-11T19:19:48.246325+0000","last_became_active":"2024-03-11T19:19:46.933557+0000","last_became_peered":"2024-03-11T19:19:46.933557+0000","last_unstale":"2024-03-11T19:19:48.246325+0000","last_undegraded":"2024-03-11T19:19:48.246325+0000","last_fullsized":"2024-03-11T19:19:48.246325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:29.227551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.237308+0000","last_change":"2024-03-11T19:19:41.402974+0000","last_active":"2024-03-11T19:20:14.237308+0000","last_peered":"2024-03-11T19:20:14.237308+0000","last_clean":"2024-03-11T19:20:14.237308+0000","last_became_active":"2024-03-11T19:19:41.402257+0000","last_became_peered":"2024-03-11T19:19:41.402257+0000","last_unstale":"2024-03-11T19:20:14.237308+0000","last_undegraded":"2024-03-11T19:20:14.237308+0000","last_fullsized":"2024-03-11T19:20:14.237308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:00:27.136592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250098+0000","last_change":"2024-03-11T19:19:42.385331+0000","last_active":"2024-03-11T19:19:48.250098+0000","last_peered":"2024-03-11T19:19:48.250098+0000","last_clean":"2024-03-11T19:19:48.250098+0000","last_became_active":"2024-03-11T19:19:42.381750+0000","last_became_peered":"2024-03-11T19:19:42.381750+0000","last_unstale":"2024-03-11T19:19:48.250098+0000","last_undegraded":"2024-03-11T19:19:48.250098+0000","last_fullsized":"2024-03-11T19:19:48.250098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:54:20.851516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243200+0000","last_change":"2024-03-11T19:19:46.801276+0000","last_active":"2024-03-11T19:19:48.243200+0000","last_peered":"2024-03-11T19:19:48.243200+0000","last_clean":"2024-03-11T19:19:48.243200+0000","last_became_active":"2024-03-11T19:19:46.800791+0000","last_became_peered":"2024-03-11T19:19:46.800791+0000","last_unstale":"2024-03-11T19:19:48.243200+0000","last_undegraded":"2024-03-11T19:19:48.243200+0000","last_fullsized":"2024-03-11T19:19:48.243200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:05.418175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.231473+0000","last_change":"2024-03-11T19:19:41.392788+0000","last_active":"2024-03-11T19:20:13.231473+0000","last_peered":"2024-03-11T19:20:13.231473+0000","last_clean":"2024-03-11T19:20:13.231473+0000","last_became_active":"2024-03-11T19:19:41.392483+0000","last_became_peered":"2024-03-11T19:19:41.392483+0000","last_unstale":"2024-03-11T19:20:13.231473+0000","last_undegraded":"2024-03-11T19:20:13.231473+0000","last_fullsized":"2024-03-11T19:20:13.231473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:55.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246121+0000","last_change":"2024-03-11T19:19:42.394123+0000","last_active":"2024-03-11T19:19:48.246121+0000","last_peered":"2024-03-11T19:19:48.246121+0000","last_clean":"2024-03-11T19:19:48.246121+0000","last_became_active":"2024-03-11T19:19:42.393785+0000","last_became_peered":"2024-03-11T19:19:42.393785+0000","last_unstale":"2024-03-11T19:19:48.246121+0000","last_undegraded":"2024-03-11T19:19:48.246121+0000","last_fullsized":"2024-03-11T19:19:48.246121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:46:24.299553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243536+0000","last_change":"2024-03-11T19:19:46.934434+0000","last_active":"2024-03-11T19:19:48.243536+0000","last_peered":"2024-03-11T19:19:48.243536+0000","last_clean":"2024-03-11T19:19:48.243536+0000","last_became_active":"2024-03-11T19:19:46.933925+0000","last_became_peered":"2024-03-11T19:19:46.933925+0000","last_unstale":"2024-03-11T19:19:48.243536+0000","last_undegraded":"2024-03-11T19:19:48.243536+0000","last_fullsized":"2024-03-11T19:19:48.243536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:33:09.098595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.231284+0000","last_change":"2024-03-11T19:19:41.394099+0000","last_active":"2024-03-11T19:20:13.231284+0000","last_peered":"2024-03-11T19:20:13.231284+0000","last_clean":"2024-03-11T19:20:13.231284+0000","last_became_active":"2024-03-11T19:19:41.393780+0000","last_became_peered":"2024-03-11T19:19:41.393780+0000","last_unstale":"2024-03-11T19:20:13.231284+0000","last_undegraded":"2024-03-11T19:20:13.231284+0000","last_fullsized":"2024-03-11T19:20:13.231284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:38:16.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243683+0000","last_change":"2024-03-11T19:19:42.363963+0000","last_active":"2024-03-11T19:19:48.243683+0000","last_peered":"2024-03-11T19:19:48.243683+0000","last_clean":"2024-03-11T19:19:48.243683+0000","last_became_active":"2024-03-11T19:19:42.363315+0000","last_became_peered":"2024-03-11T19:19:42.363315+0000","last_unstale":"2024-03-11T19:19:48.243683+0000","last_undegraded":"2024-03-11T19:19:48.243683+0000","last_fullsized":"2024-03-11T19:19:48.243683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:13:14.828230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250483+0000","last_change":"2024-03-11T19:19:46.808879+0000","last_active":"2024-03-11T19:19:48.250483+0000","last_peered":"2024-03-11T19:19:48.250483+0000","last_clean":"2024-03-11T19:19:48.250483+0000","last_became_active":"2024-03-11T19:19:46.808428+0000","last_became_peered":"2024-03-11T19:19:46.808428+0000","last_unstale":"2024-03-11T19:19:48.250483+0000","last_undegraded":"2024-03-11T19:19:48.250483+0000","last_fullsized":"2024-03-11T19:19:48.250483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:19.024854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.234510+0000","last_change":"2024-03-11T19:19:41.395487+0000","last_active":"2024-03-11T19:20:13.234510+0000","last_peered":"2024-03-11T19:20:13.234510+0000","last_clean":"2024-03-11T19:20:13.234510+0000","last_became_active":"2024-03-11T19:19:41.395081+0000","last_became_peered":"2024-03-11T19:19:41.395081+0000","last_unstale":"2024-03-11T19:20:13.234510+0000","last_undegraded":"2024-03-11T19:20:13.234510+0000","last_fullsized":"2024-03-11T19:20:13.234510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:54:58.855745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243302+0000","last_change":"2024-03-11T19:19:42.378860+0000","last_active":"2024-03-11T19:19:48.243302+0000","last_peered":"2024-03-11T19:19:48.243302+0000","last_clean":"2024-03-11T19:19:48.243302+0000","last_became_active":"2024-03-11T19:19:42.378421+0000","last_became_peered":"2024-03-11T19:19:42.378421+0000","last_unstale":"2024-03-11T19:19:48.243302+0000","last_undegraded":"2024-03-11T19:19:48.243302+0000","last_fullsized":"2024-03-11T19:19:48.243302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:08.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243115+0000","last_change":"2024-03-11T19:19:46.815770+0000","last_active":"2024-03-11T19:19:48.243115+0000","last_peered":"2024-03-11T19:19:48.243115+0000","last_clean":"2024-03-11T19:19:48.243115+0000","last_became_active":"2024-03-11T19:19:46.814982+0000","last_became_peered":"2024-03-11T19:19:46.814982+0000","last_unstale":"2024-03-11T19:19:48.243115+0000","last_undegraded":"2024-03-11T19:19:48.243115+0000","last_fullsized":"2024-03-11T19:19:48.243115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:19.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243744+0000","last_change":"2024-03-11T19:19:41.397753+0000","last_active":"2024-03-11T19:19:48.243744+0000","last_peered":"2024-03-11T19:19:48.243744+0000","last_clean":"2024-03-11T19:19:48.243744+0000","last_became_active":"2024-03-11T19:19:41.397200+0000","last_became_peered":"2024-03-11T19:19:41.397200+0000","last_unstale":"2024-03-11T19:19:48.243744+0000","last_undegraded":"2024-03-11T19:19:48.243744+0000","last_fullsized":"2024-03-11T19:19:48.243744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:38:38.326990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248865+0000","last_change":"2024-03-11T19:19:42.387285+0000","last_active":"2024-03-11T19:19:48.248865+0000","last_peered":"2024-03-11T19:19:48.248865+0000","last_clean":"2024-03-11T19:19:48.248865+0000","last_became_active":"2024-03-11T19:19:42.386919+0000","last_became_peered":"2024-03-11T19:19:42.386919+0000","last_unstale":"2024-03-11T19:19:48.248865+0000","last_undegraded":"2024-03-11T19:19:48.248865+0000","last_fullsized":"2024-03-11T19:19:48.248865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.773177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243817+0000","last_change":"2024-03-11T19:19:46.934177+0000","last_active":"2024-03-11T19:19:48.243817+0000","last_peered":"2024-03-11T19:19:48.243817+0000","last_clean":"2024-03-11T19:19:48.243817+0000","last_became_active":"2024-03-11T19:19:46.933699+0000","last_became_peered":"2024-03-11T19:19:46.933699+0000","last_unstale":"2024-03-11T19:19:48.243817+0000","last_undegraded":"2024-03-11T19:19:48.243817+0000","last_fullsized":"2024-03-11T19:19:48.243817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:09:17.433172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.230989+0000","last_change":"2024-03-11T19:19:41.396888+0000","last_active":"2024-03-11T19:20:13.230989+0000","last_peered":"2024-03-11T19:20:13.230989+0000","last_clean":"2024-03-11T19:20:13.230989+0000","last_became_active":"2024-03-11T19:19:41.396640+0000","last_became_peered":"2024-03-11T19:19:41.396640+0000","last_unstale":"2024-03-11T19:20:13.230989+0000","last_undegraded":"2024-03-11T19:20:13.230989+0000","last_fullsized":"2024-03-11T19:20:13.230989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:00.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993490+0000","last_change":"2024-03-11T19:19:42.370020+0000","last_active":"2024-03-11T19:19:47.993490+0000","last_peered":"2024-03-11T19:19:47.993490+0000","last_clean":"2024-03-11T19:19:47.993490+0000","last_became_active":"2024-03-11T19:19:42.369355+0000","last_became_peered":"2024-03-11T19:19:42.369355+0000","last_unstale":"2024-03-11T19:19:47.993490+0000","last_undegraded":"2024-03-11T19:19:47.993490+0000","last_fullsized":"2024-03-11T19:19:47.993490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:07:13.572419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788219+0000","last_change":"2024-03-11T19:19:46.818424+0000","last_active":"2024-03-11T19:19:47.788219+0000","last_peered":"2024-03-11T19:19:47.788219+0000","last_clean":"2024-03-11T19:19:47.788219+0000","last_became_active":"2024-03-11T19:19:46.817934+0000","last_became_peered":"2024-03-11T19:19:46.817934+0000","last_unstale":"2024-03-11T19:19:47.788219+0000","last_undegraded":"2024-03-11T19:19:47.788219+0000","last_fullsized":"2024-03-11T19:19:47.788219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:52:56.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244742+0000","last_change":"2024-03-11T19:19:41.399466+0000","last_active":"2024-03-11T19:19:48.244742+0000","last_peered":"2024-03-11T19:19:48.244742+0000","last_clean":"2024-03-11T19:19:48.244742+0000","last_became_active":"2024-03-11T19:19:41.399118+0000","last_became_peered":"2024-03-11T19:19:41.399118+0000","last_unstale":"2024-03-11T19:19:48.244742+0000","last_undegraded":"2024-03-11T19:19:48.244742+0000","last_fullsized":"2024-03-11T19:19:48.244742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:12:46.100596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993861+0000","last_change":"2024-03-11T19:19:42.365437+0000","last_active":"2024-03-11T19:19:47.993861+0000","last_peered":"2024-03-11T19:19:47.993861+0000","last_clean":"2024-03-11T19:19:47.993861+0000","last_became_active":"2024-03-11T19:19:42.365084+0000","last_became_peered":"2024-03-11T19:19:42.365084+0000","last_unstale":"2024-03-11T19:19:47.993861+0000","last_undegraded":"2024-03-11T19:19:47.993861+0000","last_fullsized":"2024-03-11T19:19:47.993861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:35.239100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245487+0000","last_change":"2024-03-11T19:19:46.825067+0000","last_active":"2024-03-11T19:19:48.245487+0000","last_peered":"2024-03-11T19:19:48.245487+0000","last_clean":"2024-03-11T19:19:48.245487+0000","last_became_active":"2024-03-11T19:19:46.821438+0000","last_became_peered":"2024-03-11T19:19:46.821438+0000","last_unstale":"2024-03-11T19:19:48.245487+0000","last_undegraded":"2024-03-11T19:19:48.245487+0000","last_fullsized":"2024-03-11T19:19:48.245487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:08.828669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789022+0000","last_change":"2024-03-11T19:19:41.403788+0000","last_active":"2024-03-11T19:19:47.789022+0000","last_peered":"2024-03-11T19:19:47.789022+0000","last_clean":"2024-03-11T19:19:47.789022+0000","last_became_active":"2024-03-11T19:19:41.403479+0000","last_became_peered":"2024-03-11T19:19:41.403479+0000","last_unstale":"2024-03-11T19:19:47.789022+0000","last_undegraded":"2024-03-11T19:19:47.789022+0000","last_fullsized":"2024-03-11T19:19:47.789022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:58:03.407100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993519+0000","last_change":"2024-03-11T19:19:42.381891+0000","last_active":"2024-03-11T19:19:47.993519+0000","last_peered":"2024-03-11T19:19:47.993519+0000","last_clean":"2024-03-11T19:19:47.993519+0000","last_became_active":"2024-03-11T19:19:42.381573+0000","last_became_peered":"2024-03-11T19:19:42.381573+0000","last_unstale":"2024-03-11T19:19:47.993519+0000","last_undegraded":"2024-03-11T19:19:47.993519+0000","last_fullsized":"2024-03-11T19:19:47.993519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:25.967035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788200+0000","last_change":"2024-03-11T19:19:46.803241+0000","last_active":"2024-03-11T19:19:47.788200+0000","last_peered":"2024-03-11T19:19:47.788200+0000","last_clean":"2024-03-11T19:19:47.788200+0000","last_became_active":"2024-03-11T19:19:46.802774+0000","last_became_peered":"2024-03-11T19:19:46.802774+0000","last_unstale":"2024-03-11T19:19:47.788200+0000","last_undegraded":"2024-03-11T19:19:47.788200+0000","last_fullsized":"2024-03-11T19:19:47.788200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:12.164870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792566+0000","last_change":"2024-03-11T19:19:41.394613+0000","last_active":"2024-03-11T19:19:47.792566+0000","last_peered":"2024-03-11T19:19:47.792566+0000","last_clean":"2024-03-11T19:19:47.792566+0000","last_became_active":"2024-03-11T19:19:41.394252+0000","last_became_peered":"2024-03-11T19:19:41.394252+0000","last_unstale":"2024-03-11T19:19:47.792566+0000","last_undegraded":"2024-03-11T19:19:47.792566+0000","last_fullsized":"2024-03-11T19:19:47.792566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:05:50.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993829+0000","last_change":"2024-03-11T19:19:42.366706+0000","last_active":"2024-03-11T19:19:47.993829+0000","last_peered":"2024-03-11T19:19:47.993829+0000","last_clean":"2024-03-11T19:19:47.993829+0000","last_became_active":"2024-03-11T19:19:42.366389+0000","last_became_peered":"2024-03-11T19:19:42.366389+0000","last_unstale":"2024-03-11T19:19:47.993829+0000","last_undegraded":"2024-03-11T19:19:47.993829+0000","last_fullsized":"2024-03-11T19:19:47.993829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:15:08.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243425+0000","last_change":"2024-03-11T19:19:42.367886+0000","last_active":"2024-03-11T19:19:48.243425+0000","last_peered":"2024-03-11T19:19:48.243425+0000","last_clean":"2024-03-11T19:19:48.243425+0000","last_became_active":"2024-03-11T19:19:42.367543+0000","last_became_peered":"2024-03-11T19:19:42.367543+0000","last_unstale":"2024-03-11T19:19:48.243425+0000","last_undegraded":"2024-03-11T19:19:48.243425+0000","last_fullsized":"2024-03-11T19:19:48.243425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:15:01.606205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243826+0000","last_change":"2024-03-11T19:19:42.370189+0000","last_active":"2024-03-11T19:19:48.243826+0000","last_peered":"2024-03-11T19:19:48.243826+0000","last_clean":"2024-03-11T19:19:48.243826+0000","last_became_active":"2024-03-11T19:19:42.369553+0000","last_became_peered":"2024-03-11T19:19:42.369553+0000","last_unstale":"2024-03-11T19:19:48.243826+0000","last_undegraded":"2024-03-11T19:19:48.243826+0000","last_fullsized":"2024-03-11T19:19:48.243826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:10:09.973215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243617+0000","last_change":"2024-03-11T19:19:42.380853+0000","last_active":"2024-03-11T19:19:48.243617+0000","last_peered":"2024-03-11T19:19:48.243617+0000","last_clean":"2024-03-11T19:19:48.243617+0000","last_became_active":"2024-03-11T19:19:42.380502+0000","last_became_peered":"2024-03-11T19:19:42.380502+0000","last_unstale":"2024-03-11T19:19:48.243617+0000","last_undegraded":"2024-03-11T19:19:48.243617+0000","last_fullsized":"2024-03-11T19:19:48.243617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:22:20.832801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245513+0000","last_change":"2024-03-11T19:19:42.382850+0000","last_active":"2024-03-11T19:19:48.245513+0000","last_peered":"2024-03-11T19:19:48.245513+0000","last_clean":"2024-03-11T19:19:48.245513+0000","last_became_active":"2024-03-11T19:19:42.382134+0000","last_became_peered":"2024-03-11T19:19:42.382134+0000","last_unstale":"2024-03-11T19:19:48.245513+0000","last_undegraded":"2024-03-11T19:19:48.245513+0000","last_fullsized":"2024-03-11T19:19:48.245513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:49.859290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792951+0000","last_change":"2024-03-11T19:19:42.386882+0000","last_active":"2024-03-11T19:19:47.792951+0000","last_peered":"2024-03-11T19:19:47.792951+0000","last_clean":"2024-03-11T19:19:47.792951+0000","last_became_active":"2024-03-11T19:19:42.386280+0000","last_became_peered":"2024-03-11T19:19:42.386280+0000","last_unstale":"2024-03-11T19:19:47.792951+0000","last_undegraded":"2024-03-11T19:19:47.792951+0000","last_fullsized":"2024-03-11T19:19:47.792951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:30:10.891093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244001+0000","last_change":"2024-03-11T19:19:42.368025+0000","last_active":"2024-03-11T19:19:48.244001+0000","last_peered":"2024-03-11T19:19:48.244001+0000","last_clean":"2024-03-11T19:19:48.244001+0000","last_became_active":"2024-03-11T19:19:42.367695+0000","last_became_peered":"2024-03-11T19:19:42.367695+0000","last_unstale":"2024-03-11T19:19:48.244001+0000","last_undegraded":"2024-03-11T19:19:48.244001+0000","last_fullsized":"2024-03-11T19:19:48.244001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:11:44.776358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244842+0000","last_change":"2024-03-11T19:19:42.364940+0000","last_active":"2024-03-11T19:19:48.244842+0000","last_peered":"2024-03-11T19:19:48.244842+0000","last_clean":"2024-03-11T19:19:48.244842+0000","last_became_active":"2024-03-11T19:19:42.364620+0000","last_became_peered":"2024-03-11T19:19:42.364620+0000","last_unstale":"2024-03-11T19:19:48.244842+0000","last_undegraded":"2024-03-11T19:19:48.244842+0000","last_fullsized":"2024-03-11T19:19:48.244842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:11.415716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250406+0000","last_change":"2024-03-11T19:19:42.392468+0000","last_active":"2024-03-11T19:19:48.250406+0000","last_peered":"2024-03-11T19:19:48.250406+0000","last_clean":"2024-03-11T19:19:48.250406+0000","last_became_active":"2024-03-11T19:19:42.390521+0000","last_became_peered":"2024-03-11T19:19:42.390521+0000","last_unstale":"2024-03-11T19:19:48.250406+0000","last_undegraded":"2024-03-11T19:19:48.250406+0000","last_fullsized":"2024-03-11T19:19:48.250406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:51:21.487796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788259+0000","last_change":"2024-03-11T19:19:42.393130+0000","last_active":"2024-03-11T19:19:47.788259+0000","last_peered":"2024-03-11T19:19:47.788259+0000","last_clean":"2024-03-11T19:19:47.788259+0000","last_became_active":"2024-03-11T19:19:42.392845+0000","last_became_peered":"2024-03-11T19:19:42.392845+0000","last_unstale":"2024-03-11T19:19:47.788259+0000","last_undegraded":"2024-03-11T19:19:47.788259+0000","last_fullsized":"2024-03-11T19:19:47.788259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:17:27.278574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248711+0000","last_change":"2024-03-11T19:19:42.385890+0000","last_active":"2024-03-11T19:19:48.248711+0000","last_peered":"2024-03-11T19:19:48.248711+0000","last_clean":"2024-03-11T19:19:48.248711+0000","last_became_active":"2024-03-11T19:19:42.381939+0000","last_became_peered":"2024-03-11T19:19:42.381939+0000","last_unstale":"2024-03-11T19:19:48.248711+0000","last_undegraded":"2024-03-11T19:19:48.248711+0000","last_fullsized":"2024-03-11T19:19:48.248711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:47:11.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788317+0000","last_change":"2024-03-11T19:19:42.392856+0000","last_active":"2024-03-11T19:19:47.788317+0000","last_peered":"2024-03-11T19:19:47.788317+0000","last_clean":"2024-03-11T19:19:47.788317+0000","last_became_active":"2024-03-11T19:19:42.392554+0000","last_became_peered":"2024-03-11T19:19:42.392554+0000","last_unstale":"2024-03-11T19:19:47.788317+0000","last_undegraded":"2024-03-11T19:19:47.788317+0000","last_fullsized":"2024-03-11T19:19:47.788317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:54:04.840145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.791946+0000","last_change":"2024-03-11T19:19:42.383082+0000","last_active":"2024-03-11T19:19:47.791946+0000","last_peered":"2024-03-11T19:19:47.791946+0000","last_clean":"2024-03-11T19:19:47.791946+0000","last_became_active":"2024-03-11T19:19:42.381546+0000","last_became_peered":"2024-03-11T19:19:42.381546+0000","last_unstale":"2024-03-11T19:19:47.791946+0000","last_undegraded":"2024-03-11T19:19:47.791946+0000","last_fullsized":"2024-03-11T19:19:47.791946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:20.493148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243964+0000","last_change":"2024-03-11T19:19:42.369136+0000","last_active":"2024-03-11T19:19:48.243964+0000","last_peered":"2024-03-11T19:19:48.243964+0000","last_clean":"2024-03-11T19:19:48.243964+0000","last_became_active":"2024-03-11T19:19:42.368807+0000","last_became_peered":"2024-03-11T19:19:42.368807+0000","last_unstale":"2024-03-11T19:19:48.243964+0000","last_undegraded":"2024-03-11T19:19:48.243964+0000","last_fullsized":"2024-03-11T19:19:48.243964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:16:34.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281180,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43499,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":14,"num_read_kb":48,"num_write":133,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":58491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":480845,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1100954,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481663,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":481471,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1081588,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480935,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484687,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482323,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485343,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481129,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":481489,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1072689,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:20:19.025 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:20:19.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 -- 172.21.15.72:0/4116016109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4100ff0 msgr2=0x7f17c4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 --2- 172.21.15.72:0/4116016109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4100ff0 0x7f17c4101410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009ca0 tx=0x7f17b402c100 comp rx=0 tx=0).stop 2024-03-11T19:20:19.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 -- 172.21.15.72:0/4116016109 shutdown_connections 2024-03-11T19:20:19.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 --2- 172.21.15.72:0/4116016109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4102ba0 0x7f17c4109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 --2- 172.21.15.72:0/4116016109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 0x7f17c4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 --2- 172.21.15.72:0/4116016109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4100ff0 0x7f17c4101410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 -- 172.21.15.72:0/4116016109 >> 172.21.15.72:0/4116016109 conn(0x7f17c40fc560 msgr2=0x7f17c40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:19.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 -- 172.21.15.72:0/4116016109 shutdown_connections 2024-03-11T19:20:19.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.269+0000 7f17c8a00640 1 -- 172.21.15.72:0/4116016109 wait complete. 2024-03-11T19:20:19.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.270+0000 7f17c8a00640 1 Processor -- start 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.270+0000 7f17c8a00640 1 -- start start 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 0x7f17c4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4102ba0 0x7f17c419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419ebb0 con 0x7f17c4100ff0 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419ed20 con 0x7f17c41021e0 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419eee0 con 0x7f17c4102ba0 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:19.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39902/0 (socket says 172.21.15.72:39902) 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 -- 172.21.15.72:0/1909614317 learned_addr learned my addr 172.21.15.72:0/1909614317 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17baa00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 0x7f17c4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3e00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4102ba0 0x7f17c419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 -- 172.21.15.72:0/1909614317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4102ba0 msgr2=0x7f17c419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4102ba0 0x7f17c419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 -- 172.21.15.72:0/1909614317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 msgr2=0x7f17c4197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 0x7f17c4197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.271+0000 7f17c3400640 1 -- 172.21.15.72:0/1909614317 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009490 con 0x7f17c4100ff0 2024-03-11T19:20:19.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c3400640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17b4037040 tx=0x7f17b4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4034640 con 0x7f17c4100ff0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17b4003e50 con 0x7f17c4100ff0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b4009900 con 0x7f17c4100ff0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4040450 con 0x7f17c4100ff0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.272+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c419f480 con 0x7f17c4100ff0 2024-03-11T19:20:19.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.273+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c41083e0 con 0x7f17c4100ff0 2024-03-11T19:20:19.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.275+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f17b400a040 con 0x7f17c4100ff0 2024-03-11T19:20:19.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.276+0000 7f17c0c00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 0x7f17a80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:19.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.276+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f17b40f00d0 con 0x7f17c4100ff0 2024-03-11T19:20:19.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.277+0000 7f17baa00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 0x7f17a80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:19.280 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.277+0000 7f17baa00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 0x7f17a80a3f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17c41985d0 tx=0x7f17ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:19.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.280+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17b40b8ff0 con 0x7f17c4100ff0 2024-03-11T19:20:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:19 smithi072 ceph-mon[31695]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:19 smithi072 ceph-mon[31695]: pgmap v344: 129 pgs: 129 active+clean; 619 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-03-11T19:20:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:19 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/559960991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:19.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.473+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17c402ce10 con 0x7f17a80a1ac0 2024-03-11T19:20:19.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.487+0000 7f17c0c00640 1 -- 172.21.15.72:0/1909614317 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340650 (secure 0 0 0) 0x7f17c402ce10 con 0x7f17a80a1ac0 2024-03-11T19:20:19.495 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:19.497 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 msgr2=0x7f17a80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 0x7f17a80a3f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17c41985d0 tx=0x7f17ac005f90 comp rx=0 tx=0).stop 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 msgr2=0x7f17c4197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17b4037040 tx=0x7f17b4009360 comp rx=0 tx=0).stop 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 shutdown_connections 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17a80a1ac0 0x7f17a80a3f80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c4102ba0 0x7f17c419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17c41021e0 0x7f17c4197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 --2- 172.21.15.72:0/1909614317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c4100ff0 0x7f17c4197110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:19.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.494+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 >> 172.21.15.72:0/1909614317 conn(0x7f17c40fc560 msgr2=0x7f17c41055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:19.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.495+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 shutdown_connections 2024-03-11T19:20:19.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:19.495+0000 7f17c8a00640 1 -- 172.21.15.72:0/1909614317 wait complete. 2024-03-11T19:20:19.518 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-03-11T19:20:18.066206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633984,"num_objects":58,"num_object_clones":0,"num_object_copies":175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":150,"num_read_kb":167,"num_write":315,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369240,"kb_used_data":11768,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572584,"statfs":{"total":1151940427776,"available":1151562326016,"internally_reserved":0,"allocated":12050432,"data_stored":7595116,"data_compressed":283332,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39117,"num_objects":34,"num_object_clones":0,"num_object_copies":103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":14,"num_read_kb":48,"num_write":86,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006294"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244017+0000","last_change":"2024-03-11T19:19:42.386329+0000","last_active":"2024-03-11T19:19:48.244017+0000","last_peered":"2024-03-11T19:19:48.244017+0000","last_clean":"2024-03-11T19:19:48.244017+0000","last_became_active":"2024-03-11T19:19:42.386041+0000","last_became_peered":"2024-03-11T19:19:42.386041+0000","last_unstale":"2024-03-11T19:19:48.244017+0000","last_undegraded":"2024-03-11T19:19:48.244017+0000","last_fullsized":"2024-03-11T19:19:48.244017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:05:38.258840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246437+0000","last_change":"2024-03-11T19:19:42.381508+0000","last_active":"2024-03-11T19:19:48.246437+0000","last_peered":"2024-03-11T19:19:48.246437+0000","last_clean":"2024-03-11T19:19:48.246437+0000","last_became_active":"2024-03-11T19:19:42.378340+0000","last_became_peered":"2024-03-11T19:19:42.378340+0000","last_unstale":"2024-03-11T19:19:48.246437+0000","last_undegraded":"2024-03-11T19:19:48.246437+0000","last_fullsized":"2024-03-11T19:19:48.246437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:31:06.472488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245720+0000","last_change":"2024-03-11T19:19:42.377904+0000","last_active":"2024-03-11T19:19:48.245720+0000","last_peered":"2024-03-11T19:19:48.245720+0000","last_clean":"2024-03-11T19:19:48.245720+0000","last_became_active":"2024-03-11T19:19:42.376279+0000","last_became_peered":"2024-03-11T19:19:42.376279+0000","last_unstale":"2024-03-11T19:19:48.245720+0000","last_undegraded":"2024-03-11T19:19:48.245720+0000","last_fullsized":"2024-03-11T19:19:48.245720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:52.685543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243316+0000","last_change":"2024-03-11T19:19:42.373261+0000","last_active":"2024-03-11T19:19:48.243316+0000","last_peered":"2024-03-11T19:19:48.243316+0000","last_clean":"2024-03-11T19:19:48.243316+0000","last_became_active":"2024-03-11T19:19:42.372898+0000","last_became_peered":"2024-03-11T19:19:42.372898+0000","last_unstale":"2024-03-11T19:19:48.243316+0000","last_undegraded":"2024-03-11T19:19:48.243316+0000","last_fullsized":"2024-03-11T19:19:48.243316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:22.631405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244507+0000","last_change":"2024-03-11T19:19:42.380979+0000","last_active":"2024-03-11T19:19:48.244507+0000","last_peered":"2024-03-11T19:19:48.244507+0000","last_clean":"2024-03-11T19:19:48.244507+0000","last_became_active":"2024-03-11T19:19:42.380653+0000","last_became_peered":"2024-03-11T19:19:42.380653+0000","last_unstale":"2024-03-11T19:19:48.244507+0000","last_undegraded":"2024-03-11T19:19:48.244507+0000","last_fullsized":"2024-03-11T19:19:48.244507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:39.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243766+0000","last_change":"2024-03-11T19:19:42.378182+0000","last_active":"2024-03-11T19:19:48.243766+0000","last_peered":"2024-03-11T19:19:48.243766+0000","last_clean":"2024-03-11T19:19:48.243766+0000","last_became_active":"2024-03-11T19:19:42.377810+0000","last_became_peered":"2024-03-11T19:19:42.377810+0000","last_unstale":"2024-03-11T19:19:48.243766+0000","last_undegraded":"2024-03-11T19:19:48.243766+0000","last_fullsized":"2024-03-11T19:19:48.243766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:37:20.498630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993271+0000","last_change":"2024-03-11T19:19:42.370872+0000","last_active":"2024-03-11T19:19:47.993271+0000","last_peered":"2024-03-11T19:19:47.993271+0000","last_clean":"2024-03-11T19:19:47.993271+0000","last_became_active":"2024-03-11T19:19:42.370543+0000","last_became_peered":"2024-03-11T19:19:42.370543+0000","last_unstale":"2024-03-11T19:19:47.993271+0000","last_undegraded":"2024-03-11T19:19:47.993271+0000","last_fullsized":"2024-03-11T19:19:47.993271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:48.483035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246873+0000","last_change":"2024-03-11T19:19:42.382124+0000","last_active":"2024-03-11T19:19:48.246873+0000","last_peered":"2024-03-11T19:19:48.246873+0000","last_clean":"2024-03-11T19:19:48.246873+0000","last_became_active":"2024-03-11T19:19:42.381700+0000","last_became_peered":"2024-03-11T19:19:42.381700+0000","last_unstale":"2024-03-11T19:19:48.246873+0000","last_undegraded":"2024-03-11T19:19:48.246873+0000","last_fullsized":"2024-03-11T19:19:48.246873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:21.210191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788672+0000","last_change":"2024-03-11T19:19:42.368501+0000","last_active":"2024-03-11T19:19:47.788672+0000","last_peered":"2024-03-11T19:19:47.788672+0000","last_clean":"2024-03-11T19:19:47.788672+0000","last_became_active":"2024-03-11T19:19:42.368155+0000","last_became_peered":"2024-03-11T19:19:42.368155+0000","last_unstale":"2024-03-11T19:19:47.788672+0000","last_undegraded":"2024-03-11T19:19:47.788672+0000","last_fullsized":"2024-03-11T19:19:47.788672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:55:59.244376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244282+0000","last_change":"2024-03-11T19:19:42.365113+0000","last_active":"2024-03-11T19:19:48.244282+0000","last_peered":"2024-03-11T19:19:48.244282+0000","last_clean":"2024-03-11T19:19:48.244282+0000","last_became_active":"2024-03-11T19:19:42.364710+0000","last_became_peered":"2024-03-11T19:19:42.364710+0000","last_unstale":"2024-03-11T19:19:48.244282+0000","last_undegraded":"2024-03-11T19:19:48.244282+0000","last_fullsized":"2024-03-11T19:19:48.244282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:28.999688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993815+0000","last_change":"2024-03-11T19:19:42.365977+0000","last_active":"2024-03-11T19:19:47.993815+0000","last_peered":"2024-03-11T19:19:47.993815+0000","last_clean":"2024-03-11T19:19:47.993815+0000","last_became_active":"2024-03-11T19:19:42.365220+0000","last_became_peered":"2024-03-11T19:19:42.365220+0000","last_unstale":"2024-03-11T19:19:47.993815+0000","last_undegraded":"2024-03-11T19:19:47.993815+0000","last_fullsized":"2024-03-11T19:19:47.993815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:45.707973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243194+0000","last_change":"2024-03-11T19:19:42.368596+0000","last_active":"2024-03-11T19:19:48.243194+0000","last_peered":"2024-03-11T19:19:48.243194+0000","last_clean":"2024-03-11T19:19:48.243194+0000","last_became_active":"2024-03-11T19:19:42.366526+0000","last_became_peered":"2024-03-11T19:19:42.366526+0000","last_unstale":"2024-03-11T19:19:48.243194+0000","last_undegraded":"2024-03-11T19:19:48.243194+0000","last_fullsized":"2024-03-11T19:19:48.243194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:57.992445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792062+0000","last_change":"2024-03-11T19:19:42.374433+0000","last_active":"2024-03-11T19:19:47.792062+0000","last_peered":"2024-03-11T19:19:47.792062+0000","last_clean":"2024-03-11T19:19:47.792062+0000","last_became_active":"2024-03-11T19:19:42.371924+0000","last_became_peered":"2024-03-11T19:19:42.371924+0000","last_unstale":"2024-03-11T19:19:47.792062+0000","last_undegraded":"2024-03-11T19:19:47.792062+0000","last_fullsized":"2024-03-11T19:19:47.792062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:37.515381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245108+0000","last_change":"2024-03-11T19:19:42.378885+0000","last_active":"2024-03-11T19:19:48.245108+0000","last_peered":"2024-03-11T19:19:48.245108+0000","last_clean":"2024-03-11T19:19:48.245108+0000","last_became_active":"2024-03-11T19:19:42.378505+0000","last_became_peered":"2024-03-11T19:19:42.378505+0000","last_unstale":"2024-03-11T19:19:48.245108+0000","last_undegraded":"2024-03-11T19:19:48.245108+0000","last_fullsized":"2024-03-11T19:19:48.245108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:12.380431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244935+0000","last_change":"2024-03-11T19:19:42.377500+0000","last_active":"2024-03-11T19:19:48.244935+0000","last_peered":"2024-03-11T19:19:48.244935+0000","last_clean":"2024-03-11T19:19:48.244935+0000","last_became_active":"2024-03-11T19:19:42.377083+0000","last_became_peered":"2024-03-11T19:19:42.377083+0000","last_unstale":"2024-03-11T19:19:48.244935+0000","last_undegraded":"2024-03-11T19:19:48.244935+0000","last_fullsized":"2024-03-11T19:19:48.244935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:02:56.146071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788080+0000","last_change":"2024-03-11T19:19:42.375809+0000","last_active":"2024-03-11T19:19:47.788080+0000","last_peered":"2024-03-11T19:19:47.788080+0000","last_clean":"2024-03-11T19:19:47.788080+0000","last_became_active":"2024-03-11T19:19:42.375422+0000","last_became_peered":"2024-03-11T19:19:42.375422+0000","last_unstale":"2024-03-11T19:19:47.788080+0000","last_undegraded":"2024-03-11T19:19:47.788080+0000","last_fullsized":"2024-03-11T19:19:47.788080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:17:21.344587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243732+0000","last_change":"2024-03-11T19:19:42.381932+0000","last_active":"2024-03-11T19:19:48.243732+0000","last_peered":"2024-03-11T19:19:48.243732+0000","last_clean":"2024-03-11T19:19:48.243732+0000","last_became_active":"2024-03-11T19:19:42.381591+0000","last_became_peered":"2024-03-11T19:19:42.381591+0000","last_unstale":"2024-03-11T19:19:48.243732+0000","last_undegraded":"2024-03-11T19:19:48.243732+0000","last_fullsized":"2024-03-11T19:19:48.243732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:03.619050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249551+0000","last_change":"2024-03-11T19:19:42.374924+0000","last_active":"2024-03-11T19:19:48.249551+0000","last_peered":"2024-03-11T19:19:48.249551+0000","last_clean":"2024-03-11T19:19:48.249551+0000","last_became_active":"2024-03-11T19:19:42.374094+0000","last_became_peered":"2024-03-11T19:19:42.374094+0000","last_unstale":"2024-03-11T19:19:48.249551+0000","last_undegraded":"2024-03-11T19:19:48.249551+0000","last_fullsized":"2024-03-11T19:19:48.249551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:15.327665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244606+0000","last_change":"2024-03-11T19:19:42.381807+0000","last_active":"2024-03-11T19:19:48.244606+0000","last_peered":"2024-03-11T19:19:48.244606+0000","last_clean":"2024-03-11T19:19:48.244606+0000","last_became_active":"2024-03-11T19:19:42.381056+0000","last_became_peered":"2024-03-11T19:19:42.381056+0000","last_unstale":"2024-03-11T19:19:48.244606+0000","last_undegraded":"2024-03-11T19:19:48.244606+0000","last_fullsized":"2024-03-11T19:19:48.244606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:00.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243693+0000","last_change":"2024-03-11T19:19:46.825875+0000","last_active":"2024-03-11T19:19:48.243693+0000","last_peered":"2024-03-11T19:19:48.243693+0000","last_clean":"2024-03-11T19:19:48.243693+0000","last_became_active":"2024-03-11T19:19:46.825528+0000","last_became_peered":"2024-03-11T19:19:46.825528+0000","last_unstale":"2024-03-11T19:19:48.243693+0000","last_undegraded":"2024-03-11T19:19:48.243693+0000","last_fullsized":"2024-03-11T19:19:48.243693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:32:38.667775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.240411+0000","last_change":"2024-03-11T19:19:41.393633+0000","last_active":"2024-03-11T19:20:14.240411+0000","last_peered":"2024-03-11T19:20:14.240411+0000","last_clean":"2024-03-11T19:20:14.240411+0000","last_became_active":"2024-03-11T19:19:41.389865+0000","last_became_peered":"2024-03-11T19:19:41.389865+0000","last_unstale":"2024-03-11T19:20:14.240411+0000","last_undegraded":"2024-03-11T19:20:14.240411+0000","last_fullsized":"2024-03-11T19:20:14.240411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:29.251688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249750+0000","last_change":"2024-03-11T19:19:42.392393+0000","last_active":"2024-03-11T19:19:48.249750+0000","last_peered":"2024-03-11T19:19:48.249750+0000","last_clean":"2024-03-11T19:19:48.249750+0000","last_became_active":"2024-03-11T19:19:42.390724+0000","last_became_peered":"2024-03-11T19:19:42.390724+0000","last_unstale":"2024-03-11T19:19:48.249750+0000","last_undegraded":"2024-03-11T19:19:48.249750+0000","last_fullsized":"2024-03-11T19:19:48.249750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:49.410831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.242989+0000","last_change":"2024-03-11T19:19:46.817231+0000","last_active":"2024-03-11T19:19:48.242989+0000","last_peered":"2024-03-11T19:19:48.242989+0000","last_clean":"2024-03-11T19:19:48.242989+0000","last_became_active":"2024-03-11T19:19:46.816818+0000","last_became_peered":"2024-03-11T19:19:46.816818+0000","last_unstale":"2024-03-11T19:19:48.242989+0000","last_undegraded":"2024-03-11T19:19:48.242989+0000","last_fullsized":"2024-03-11T19:19:48.242989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:49.562628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249857+0000","last_change":"2024-03-11T19:19:41.401029+0000","last_active":"2024-03-11T19:19:48.249857+0000","last_peered":"2024-03-11T19:19:48.249857+0000","last_clean":"2024-03-11T19:19:48.249857+0000","last_became_active":"2024-03-11T19:19:41.400659+0000","last_became_peered":"2024-03-11T19:19:41.400659+0000","last_unstale":"2024-03-11T19:19:48.249857+0000","last_undegraded":"2024-03-11T19:19:48.249857+0000","last_fullsized":"2024-03-11T19:19:48.249857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:07.903543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246739+0000","last_change":"2024-03-11T19:19:42.380714+0000","last_active":"2024-03-11T19:19:48.246739+0000","last_peered":"2024-03-11T19:19:48.246739+0000","last_clean":"2024-03-11T19:19:48.246739+0000","last_became_active":"2024-03-11T19:19:42.380064+0000","last_became_peered":"2024-03-11T19:19:42.380064+0000","last_unstale":"2024-03-11T19:19:48.246739+0000","last_undegraded":"2024-03-11T19:19:48.246739+0000","last_fullsized":"2024-03-11T19:19:48.246739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:47.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243622+0000","last_change":"2024-03-11T19:19:46.815643+0000","last_active":"2024-03-11T19:19:48.243622+0000","last_peered":"2024-03-11T19:19:48.243622+0000","last_clean":"2024-03-11T19:19:48.243622+0000","last_became_active":"2024-03-11T19:19:46.815222+0000","last_became_peered":"2024-03-11T19:19:46.815222+0000","last_unstale":"2024-03-11T19:19:48.243622+0000","last_undegraded":"2024-03-11T19:19:48.243622+0000","last_fullsized":"2024-03-11T19:19:48.243622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:44:24.625495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.235525+0000","last_change":"2024-03-11T19:19:41.397810+0000","last_active":"2024-03-11T19:20:14.235525+0000","last_peered":"2024-03-11T19:20:14.235525+0000","last_clean":"2024-03-11T19:20:14.235525+0000","last_became_active":"2024-03-11T19:19:41.396726+0000","last_became_peered":"2024-03-11T19:19:41.396726+0000","last_unstale":"2024-03-11T19:20:14.235525+0000","last_undegraded":"2024-03-11T19:20:14.235525+0000","last_fullsized":"2024-03-11T19:20:14.235525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:26.328409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788336+0000","last_change":"2024-03-11T19:19:42.360595+0000","last_active":"2024-03-11T19:19:47.788336+0000","last_peered":"2024-03-11T19:19:47.788336+0000","last_clean":"2024-03-11T19:19:47.788336+0000","last_became_active":"2024-03-11T19:19:42.360230+0000","last_became_peered":"2024-03-11T19:19:42.360230+0000","last_unstale":"2024-03-11T19:19:47.788336+0000","last_undegraded":"2024-03-11T19:19:47.788336+0000","last_fullsized":"2024-03-11T19:19:47.788336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:45.480391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.242931+0000","last_change":"2024-03-11T19:19:46.817442+0000","last_active":"2024-03-11T19:19:48.242931+0000","last_peered":"2024-03-11T19:19:48.242931+0000","last_clean":"2024-03-11T19:19:48.242931+0000","last_became_active":"2024-03-11T19:19:46.816944+0000","last_became_peered":"2024-03-11T19:19:46.816944+0000","last_unstale":"2024-03-11T19:19:48.242931+0000","last_undegraded":"2024-03-11T19:19:48.242931+0000","last_fullsized":"2024-03-11T19:19:48.242931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:05:43.783450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243550+0000","last_change":"2024-03-11T19:19:41.389608+0000","last_active":"2024-03-11T19:19:48.243550+0000","last_peered":"2024-03-11T19:19:48.243550+0000","last_clean":"2024-03-11T19:19:48.243550+0000","last_became_active":"2024-03-11T19:19:41.389281+0000","last_became_peered":"2024-03-11T19:19:41.389281+0000","last_unstale":"2024-03-11T19:19:48.243550+0000","last_undegraded":"2024-03-11T19:19:48.243550+0000","last_fullsized":"2024-03-11T19:19:48.243550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:46.213494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788893+0000","last_change":"2024-03-11T19:19:42.371444+0000","last_active":"2024-03-11T19:19:47.788893+0000","last_peered":"2024-03-11T19:19:47.788893+0000","last_clean":"2024-03-11T19:19:47.788893+0000","last_became_active":"2024-03-11T19:19:42.371129+0000","last_became_peered":"2024-03-11T19:19:42.371129+0000","last_unstale":"2024-03-11T19:19:47.788893+0000","last_undegraded":"2024-03-11T19:19:47.788893+0000","last_fullsized":"2024-03-11T19:19:47.788893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:04:07.449453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244518+0000","last_change":"2024-03-11T19:19:46.805616+0000","last_active":"2024-03-11T19:19:48.244518+0000","last_peered":"2024-03-11T19:19:48.244518+0000","last_clean":"2024-03-11T19:19:48.244518+0000","last_became_active":"2024-03-11T19:19:46.805069+0000","last_became_peered":"2024-03-11T19:19:46.805069+0000","last_unstale":"2024-03-11T19:19:48.244518+0000","last_undegraded":"2024-03-11T19:19:48.244518+0000","last_fullsized":"2024-03-11T19:19:48.244518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:47:23.197283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232608+0000","last_change":"2024-03-11T19:19:41.392375+0000","last_active":"2024-03-11T19:20:13.232608+0000","last_peered":"2024-03-11T19:20:13.232608+0000","last_clean":"2024-03-11T19:20:13.232608+0000","last_became_active":"2024-03-11T19:19:41.392057+0000","last_became_peered":"2024-03-11T19:19:41.392057+0000","last_unstale":"2024-03-11T19:20:13.232608+0000","last_undegraded":"2024-03-11T19:20:13.232608+0000","last_fullsized":"2024-03-11T19:20:13.232608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:12.842671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244284+0000","last_change":"2024-03-11T19:19:42.373255+0000","last_active":"2024-03-11T19:19:48.244284+0000","last_peered":"2024-03-11T19:19:48.244284+0000","last_clean":"2024-03-11T19:19:48.244284+0000","last_became_active":"2024-03-11T19:19:42.372900+0000","last_became_peered":"2024-03-11T19:19:42.372900+0000","last_unstale":"2024-03-11T19:19:48.244284+0000","last_undegraded":"2024-03-11T19:19:48.244284+0000","last_fullsized":"2024-03-11T19:19:48.244284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:56:45.517189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244494+0000","last_change":"2024-03-11T19:19:46.819292+0000","last_active":"2024-03-11T19:19:48.244494+0000","last_peered":"2024-03-11T19:19:48.244494+0000","last_clean":"2024-03-11T19:19:48.244494+0000","last_became_active":"2024-03-11T19:19:46.818932+0000","last_became_peered":"2024-03-11T19:19:46.818932+0000","last_unstale":"2024-03-11T19:19:48.244494+0000","last_undegraded":"2024-03-11T19:19:48.244494+0000","last_fullsized":"2024-03-11T19:19:48.244494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:18:59.568422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.234747+0000","last_change":"2024-03-11T19:19:41.385591+0000","last_active":"2024-03-11T19:20:14.234747+0000","last_peered":"2024-03-11T19:20:14.234747+0000","last_clean":"2024-03-11T19:20:14.234747+0000","last_became_active":"2024-03-11T19:19:41.385236+0000","last_became_peered":"2024-03-11T19:19:41.385236+0000","last_unstale":"2024-03-11T19:20:14.234747+0000","last_undegraded":"2024-03-11T19:20:14.234747+0000","last_fullsized":"2024-03-11T19:20:14.234747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:55:55.558948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788683+0000","last_change":"2024-03-11T19:19:42.380934+0000","last_active":"2024-03-11T19:19:47.788683+0000","last_peered":"2024-03-11T19:19:47.788683+0000","last_clean":"2024-03-11T19:19:47.788683+0000","last_became_active":"2024-03-11T19:19:42.380602+0000","last_became_peered":"2024-03-11T19:19:42.380602+0000","last_unstale":"2024-03-11T19:19:47.788683+0000","last_undegraded":"2024-03-11T19:19:47.788683+0000","last_fullsized":"2024-03-11T19:19:47.788683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:58:26.813552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243663+0000","last_change":"2024-03-11T19:19:46.814227+0000","last_active":"2024-03-11T19:19:48.243663+0000","last_peered":"2024-03-11T19:19:48.243663+0000","last_clean":"2024-03-11T19:19:48.243663+0000","last_became_active":"2024-03-11T19:19:46.813874+0000","last_became_peered":"2024-03-11T19:19:46.813874+0000","last_unstale":"2024-03-11T19:19:48.243663+0000","last_undegraded":"2024-03-11T19:19:48.243663+0000","last_fullsized":"2024-03-11T19:19:48.243663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:21.493313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.242599+0000","last_change":"2024-03-11T19:19:41.392245+0000","last_active":"2024-03-11T19:20:14.242599+0000","last_peered":"2024-03-11T19:20:14.242599+0000","last_clean":"2024-03-11T19:20:14.242599+0000","last_became_active":"2024-03-11T19:19:41.391928+0000","last_became_peered":"2024-03-11T19:19:41.391928+0000","last_unstale":"2024-03-11T19:20:14.242599+0000","last_undegraded":"2024-03-11T19:20:14.242599+0000","last_fullsized":"2024-03-11T19:20:14.242599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:20.556826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246600+0000","last_change":"2024-03-11T19:19:42.389431+0000","last_active":"2024-03-11T19:19:48.246600+0000","last_peered":"2024-03-11T19:19:48.246600+0000","last_clean":"2024-03-11T19:19:48.246600+0000","last_became_active":"2024-03-11T19:19:42.387885+0000","last_became_peered":"2024-03-11T19:19:42.387885+0000","last_unstale":"2024-03-11T19:19:48.246600+0000","last_undegraded":"2024-03-11T19:19:48.246600+0000","last_fullsized":"2024-03-11T19:19:48.246600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:43.753079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245978+0000","last_change":"2024-03-11T19:19:46.823474+0000","last_active":"2024-03-11T19:19:48.245978+0000","last_peered":"2024-03-11T19:19:48.245978+0000","last_clean":"2024-03-11T19:19:48.245978+0000","last_became_active":"2024-03-11T19:19:46.823008+0000","last_became_peered":"2024-03-11T19:19:46.823008+0000","last_unstale":"2024-03-11T19:19:48.245978+0000","last_undegraded":"2024-03-11T19:19:48.245978+0000","last_fullsized":"2024-03-11T19:19:48.245978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:28.381809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788180+0000","last_change":"2024-03-11T19:19:41.395332+0000","last_active":"2024-03-11T19:19:47.788180+0000","last_peered":"2024-03-11T19:19:47.788180+0000","last_clean":"2024-03-11T19:19:47.788180+0000","last_became_active":"2024-03-11T19:19:41.394989+0000","last_became_peered":"2024-03-11T19:19:41.394989+0000","last_unstale":"2024-03-11T19:19:47.788180+0000","last_undegraded":"2024-03-11T19:19:47.788180+0000","last_fullsized":"2024-03-11T19:19:47.788180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:57:31.486891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.247071+0000","last_change":"2024-03-11T19:19:42.393223+0000","last_active":"2024-03-11T19:19:48.247071+0000","last_peered":"2024-03-11T19:19:48.247071+0000","last_clean":"2024-03-11T19:19:48.247071+0000","last_became_active":"2024-03-11T19:19:42.392787+0000","last_became_peered":"2024-03-11T19:19:42.392787+0000","last_unstale":"2024-03-11T19:19:48.247071+0000","last_undegraded":"2024-03-11T19:19:48.247071+0000","last_fullsized":"2024-03-11T19:19:48.247071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:32:50.693896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246192+0000","last_change":"2024-03-11T19:19:46.808305+0000","last_active":"2024-03-11T19:19:48.246192+0000","last_peered":"2024-03-11T19:19:48.246192+0000","last_clean":"2024-03-11T19:19:48.246192+0000","last_became_active":"2024-03-11T19:19:46.807816+0000","last_became_peered":"2024-03-11T19:19:46.807816+0000","last_unstale":"2024-03-11T19:19:48.246192+0000","last_undegraded":"2024-03-11T19:19:48.246192+0000","last_fullsized":"2024-03-11T19:19:48.246192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:55:43.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:16.245590+0000","last_change":"2024-03-11T19:19:41.389335+0000","last_active":"2024-03-11T19:20:16.245590+0000","last_peered":"2024-03-11T19:20:16.245590+0000","last_clean":"2024-03-11T19:20:16.245590+0000","last_became_active":"2024-03-11T19:19:41.389023+0000","last_became_peered":"2024-03-11T19:19:41.389023+0000","last_unstale":"2024-03-11T19:20:16.245590+0000","last_undegraded":"2024-03-11T19:20:16.245590+0000","last_fullsized":"2024-03-11T19:20:16.245590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:12:59.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788567+0000","last_change":"2024-03-11T19:19:42.378379+0000","last_active":"2024-03-11T19:19:47.788567+0000","last_peered":"2024-03-11T19:19:47.788567+0000","last_clean":"2024-03-11T19:19:47.788567+0000","last_became_active":"2024-03-11T19:19:42.378033+0000","last_became_peered":"2024-03-11T19:19:42.378033+0000","last_unstale":"2024-03-11T19:19:47.788567+0000","last_undegraded":"2024-03-11T19:19:47.788567+0000","last_fullsized":"2024-03-11T19:19:47.788567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:24.154420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249448+0000","last_change":"2024-03-11T19:19:46.809464+0000","last_active":"2024-03-11T19:19:48.249448+0000","last_peered":"2024-03-11T19:19:48.249448+0000","last_clean":"2024-03-11T19:19:48.249448+0000","last_became_active":"2024-03-11T19:19:46.808955+0000","last_became_peered":"2024-03-11T19:19:46.808955+0000","last_unstale":"2024-03-11T19:19:48.249448+0000","last_undegraded":"2024-03-11T19:19:48.249448+0000","last_fullsized":"2024-03-11T19:19:48.249448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:42.088992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232018+0000","last_change":"2024-03-11T19:19:41.395619+0000","last_active":"2024-03-11T19:20:13.232018+0000","last_peered":"2024-03-11T19:20:13.232018+0000","last_clean":"2024-03-11T19:20:13.232018+0000","last_became_active":"2024-03-11T19:19:41.394658+0000","last_became_peered":"2024-03-11T19:19:41.394658+0000","last_unstale":"2024-03-11T19:20:13.232018+0000","last_undegraded":"2024-03-11T19:20:13.232018+0000","last_fullsized":"2024-03-11T19:20:13.232018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:10:39.020508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.247467+0000","last_change":"2024-03-11T19:19:42.380416+0000","last_active":"2024-03-11T19:19:48.247467+0000","last_peered":"2024-03-11T19:19:48.247467+0000","last_clean":"2024-03-11T19:19:48.247467+0000","last_became_active":"2024-03-11T19:19:42.379625+0000","last_became_peered":"2024-03-11T19:19:42.379625+0000","last_unstale":"2024-03-11T19:19:48.247467+0000","last_undegraded":"2024-03-11T19:19:48.247467+0000","last_fullsized":"2024-03-11T19:19:48.247467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:45:45.374211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243508+0000","last_change":"2024-03-11T19:19:46.808005+0000","last_active":"2024-03-11T19:19:48.243508+0000","last_peered":"2024-03-11T19:19:48.243508+0000","last_clean":"2024-03-11T19:19:48.243508+0000","last_became_active":"2024-03-11T19:19:46.807549+0000","last_became_peered":"2024-03-11T19:19:46.807549+0000","last_unstale":"2024-03-11T19:19:48.243508+0000","last_undegraded":"2024-03-11T19:19:48.243508+0000","last_fullsized":"2024-03-11T19:19:48.243508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:17.171846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789063+0000","last_change":"2024-03-11T19:19:41.398755+0000","last_active":"2024-03-11T19:19:47.789063+0000","last_peered":"2024-03-11T19:19:47.789063+0000","last_clean":"2024-03-11T19:19:47.789063+0000","last_became_active":"2024-03-11T19:19:41.398406+0000","last_became_peered":"2024-03-11T19:19:41.398406+0000","last_unstale":"2024-03-11T19:19:47.789063+0000","last_undegraded":"2024-03-11T19:19:47.789063+0000","last_fullsized":"2024-03-11T19:19:47.789063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:37.793328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788502+0000","last_change":"2024-03-11T19:19:42.364757+0000","last_active":"2024-03-11T19:19:47.788502+0000","last_peered":"2024-03-11T19:19:47.788502+0000","last_clean":"2024-03-11T19:19:47.788502+0000","last_became_active":"2024-03-11T19:19:42.364432+0000","last_became_peered":"2024-03-11T19:19:42.364432+0000","last_unstale":"2024-03-11T19:19:47.788502+0000","last_undegraded":"2024-03-11T19:19:47.788502+0000","last_fullsized":"2024-03-11T19:19:47.788502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:29.226200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246975+0000","last_change":"2024-03-11T19:19:46.809300+0000","last_active":"2024-03-11T19:19:48.246975+0000","last_peered":"2024-03-11T19:19:48.246975+0000","last_clean":"2024-03-11T19:19:48.246975+0000","last_became_active":"2024-03-11T19:19:46.807327+0000","last_became_peered":"2024-03-11T19:19:46.807327+0000","last_unstale":"2024-03-11T19:19:48.246975+0000","last_undegraded":"2024-03-11T19:19:48.246975+0000","last_fullsized":"2024-03-11T19:19:48.246975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:37:54.082436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'25","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:17.250107+0000","last_change":"2024-03-11T19:19:41.392270+0000","last_active":"2024-03-11T19:20:17.250107+0000","last_peered":"2024-03-11T19:20:17.250107+0000","last_clean":"2024-03-11T19:20:17.250107+0000","last_became_active":"2024-03-11T19:19:41.391559+0000","last_became_peered":"2024-03-11T19:19:41.391559+0000","last_unstale":"2024-03-11T19:20:17.250107+0000","last_undegraded":"2024-03-11T19:20:17.250107+0000","last_fullsized":"2024-03-11T19:20:17.250107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:45.655610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":44,"num_write":26,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244864+0000","last_change":"2024-03-11T19:19:42.368323+0000","last_active":"2024-03-11T19:19:48.244864+0000","last_peered":"2024-03-11T19:19:48.244864+0000","last_clean":"2024-03-11T19:19:48.244864+0000","last_became_active":"2024-03-11T19:19:42.367973+0000","last_became_peered":"2024-03-11T19:19:42.367973+0000","last_unstale":"2024-03-11T19:19:48.244864+0000","last_undegraded":"2024-03-11T19:19:48.244864+0000","last_fullsized":"2024-03-11T19:19:48.244864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:11.573348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789347+0000","last_change":"2024-03-11T19:19:46.822255+0000","last_active":"2024-03-11T19:19:47.789347+0000","last_peered":"2024-03-11T19:19:47.789347+0000","last_clean":"2024-03-11T19:19:47.789347+0000","last_became_active":"2024-03-11T19:19:46.821306+0000","last_became_peered":"2024-03-11T19:19:46.821306+0000","last_unstale":"2024-03-11T19:19:47.789347+0000","last_undegraded":"2024-03-11T19:19:47.789347+0000","last_fullsized":"2024-03-11T19:19:47.789347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:22.353676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.232403+0000","last_change":"2024-03-11T19:19:41.397723+0000","last_active":"2024-03-11T19:20:13.232403+0000","last_peered":"2024-03-11T19:20:13.232403+0000","last_clean":"2024-03-11T19:20:13.232403+0000","last_became_active":"2024-03-11T19:19:41.397394+0000","last_became_peered":"2024-03-11T19:19:41.397394+0000","last_unstale":"2024-03-11T19:20:13.232403+0000","last_undegraded":"2024-03-11T19:20:13.232403+0000","last_fullsized":"2024-03-11T19:20:13.232403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:48:42.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792126+0000","last_change":"2024-03-11T19:19:42.389254+0000","last_active":"2024-03-11T19:19:47.792126+0000","last_peered":"2024-03-11T19:19:47.792126+0000","last_clean":"2024-03-11T19:19:47.792126+0000","last_became_active":"2024-03-11T19:19:42.388861+0000","last_became_peered":"2024-03-11T19:19:42.388861+0000","last_unstale":"2024-03-11T19:19:47.792126+0000","last_undegraded":"2024-03-11T19:19:47.792126+0000","last_fullsized":"2024-03-11T19:19:47.792126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:03.308802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243947+0000","last_change":"2024-03-11T19:19:46.821667+0000","last_active":"2024-03-11T19:19:48.243947+0000","last_peered":"2024-03-11T19:19:48.243947+0000","last_clean":"2024-03-11T19:19:48.243947+0000","last_became_active":"2024-03-11T19:19:46.821238+0000","last_became_peered":"2024-03-11T19:19:46.821238+0000","last_unstale":"2024-03-11T19:19:48.243947+0000","last_undegraded":"2024-03-11T19:19:48.243947+0000","last_fullsized":"2024-03-11T19:19:48.243947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:30.997248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.233217+0000","last_change":"2024-03-11T19:19:41.403277+0000","last_active":"2024-03-11T19:20:13.233217+0000","last_peered":"2024-03-11T19:20:13.233217+0000","last_clean":"2024-03-11T19:20:13.233217+0000","last_became_active":"2024-03-11T19:19:41.402875+0000","last_became_peered":"2024-03-11T19:19:41.402875+0000","last_unstale":"2024-03-11T19:20:13.233217+0000","last_undegraded":"2024-03-11T19:20:13.233217+0000","last_fullsized":"2024-03-11T19:20:13.233217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.937257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993609+0000","last_change":"2024-03-11T19:19:42.376244+0000","last_active":"2024-03-11T19:19:47.993609+0000","last_peered":"2024-03-11T19:19:47.993609+0000","last_clean":"2024-03-11T19:19:47.993609+0000","last_became_active":"2024-03-11T19:19:42.375896+0000","last_became_peered":"2024-03-11T19:19:42.375896+0000","last_unstale":"2024-03-11T19:19:47.993609+0000","last_undegraded":"2024-03-11T19:19:47.993609+0000","last_fullsized":"2024-03-11T19:19:47.993609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:46.295898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249767+0000","last_change":"2024-03-11T19:19:46.811977+0000","last_active":"2024-03-11T19:19:48.249767+0000","last_peered":"2024-03-11T19:19:48.249767+0000","last_clean":"2024-03-11T19:19:48.249767+0000","last_became_active":"2024-03-11T19:19:46.811570+0000","last_became_peered":"2024-03-11T19:19:46.811570+0000","last_unstale":"2024-03-11T19:19:48.249767+0000","last_undegraded":"2024-03-11T19:19:48.249767+0000","last_fullsized":"2024-03-11T19:19:48.249767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:00:18.329040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.235103+0000","last_change":"2024-03-11T19:19:41.387779+0000","last_active":"2024-03-11T19:20:14.235103+0000","last_peered":"2024-03-11T19:20:14.235103+0000","last_clean":"2024-03-11T19:20:14.235103+0000","last_became_active":"2024-03-11T19:19:41.387424+0000","last_became_peered":"2024-03-11T19:19:41.387424+0000","last_unstale":"2024-03-11T19:20:14.235103+0000","last_undegraded":"2024-03-11T19:20:14.235103+0000","last_fullsized":"2024-03-11T19:20:14.235103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:49.936069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993327+0000","last_change":"2024-03-11T19:19:42.367664+0000","last_active":"2024-03-11T19:19:47.993327+0000","last_peered":"2024-03-11T19:19:47.993327+0000","last_clean":"2024-03-11T19:19:47.993327+0000","last_became_active":"2024-03-11T19:19:42.367343+0000","last_became_peered":"2024-03-11T19:19:42.367343+0000","last_unstale":"2024-03-11T19:19:47.993327+0000","last_undegraded":"2024-03-11T19:19:47.993327+0000","last_fullsized":"2024-03-11T19:19:47.993327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:29.215094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792428+0000","last_change":"2024-03-11T19:19:46.821832+0000","last_active":"2024-03-11T19:19:47.792428+0000","last_peered":"2024-03-11T19:19:47.792428+0000","last_clean":"2024-03-11T19:19:47.792428+0000","last_became_active":"2024-03-11T19:19:46.821183+0000","last_became_peered":"2024-03-11T19:19:46.821183+0000","last_unstale":"2024-03-11T19:19:47.792428+0000","last_undegraded":"2024-03-11T19:19:47.792428+0000","last_fullsized":"2024-03-11T19:19:47.792428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:56.293370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.240574+0000","last_change":"2024-03-11T19:19:41.400790+0000","last_active":"2024-03-11T19:20:14.240574+0000","last_peered":"2024-03-11T19:20:14.240574+0000","last_clean":"2024-03-11T19:20:14.240574+0000","last_became_active":"2024-03-11T19:19:41.400456+0000","last_became_peered":"2024-03-11T19:19:41.400456+0000","last_unstale":"2024-03-11T19:20:14.240574+0000","last_undegraded":"2024-03-11T19:20:14.240574+0000","last_fullsized":"2024-03-11T19:20:14.240574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:00:18.195113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788315+0000","last_change":"2024-03-11T19:19:42.370608+0000","last_active":"2024-03-11T19:19:47.788315+0000","last_peered":"2024-03-11T19:19:47.788315+0000","last_clean":"2024-03-11T19:19:47.788315+0000","last_became_active":"2024-03-11T19:19:42.370259+0000","last_became_peered":"2024-03-11T19:19:42.370259+0000","last_unstale":"2024-03-11T19:19:47.788315+0000","last_undegraded":"2024-03-11T19:19:47.788315+0000","last_fullsized":"2024-03-11T19:19:47.788315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:50:05.778074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788431+0000","last_change":"2024-03-11T19:19:46.826437+0000","last_active":"2024-03-11T19:19:47.788431+0000","last_peered":"2024-03-11T19:19:47.788431+0000","last_clean":"2024-03-11T19:19:47.788431+0000","last_became_active":"2024-03-11T19:19:46.825964+0000","last_became_peered":"2024-03-11T19:19:46.825964+0000","last_unstale":"2024-03-11T19:19:47.788431+0000","last_undegraded":"2024-03-11T19:19:47.788431+0000","last_fullsized":"2024-03-11T19:19:47.788431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:07.970696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244196+0000","last_change":"2024-03-11T19:19:41.388747+0000","last_active":"2024-03-11T19:19:48.244196+0000","last_peered":"2024-03-11T19:19:48.244196+0000","last_clean":"2024-03-11T19:19:48.244196+0000","last_became_active":"2024-03-11T19:19:41.388408+0000","last_became_peered":"2024-03-11T19:19:41.388408+0000","last_unstale":"2024-03-11T19:19:48.244196+0000","last_undegraded":"2024-03-11T19:19:48.244196+0000","last_fullsized":"2024-03-11T19:19:48.244196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:32:20.195039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244109+0000","last_change":"2024-03-11T19:19:42.371432+0000","last_active":"2024-03-11T19:19:48.244109+0000","last_peered":"2024-03-11T19:19:48.244109+0000","last_clean":"2024-03-11T19:19:48.244109+0000","last_became_active":"2024-03-11T19:19:42.371084+0000","last_became_peered":"2024-03-11T19:19:42.371084+0000","last_unstale":"2024-03-11T19:19:48.244109+0000","last_undegraded":"2024-03-11T19:19:48.244109+0000","last_fullsized":"2024-03-11T19:19:48.244109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:10:50.386891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244395+0000","last_change":"2024-03-11T19:19:46.934632+0000","last_active":"2024-03-11T19:19:48.244395+0000","last_peered":"2024-03-11T19:19:48.244395+0000","last_clean":"2024-03-11T19:19:48.244395+0000","last_became_active":"2024-03-11T19:19:46.934172+0000","last_became_peered":"2024-03-11T19:19:46.934172+0000","last_unstale":"2024-03-11T19:19:48.244395+0000","last_undegraded":"2024-03-11T19:19:48.244395+0000","last_fullsized":"2024-03-11T19:19:48.244395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:01:21.675154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.249602+0000","last_change":"2024-03-11T19:19:41.397599+0000","last_active":"2024-03-11T19:19:48.249602+0000","last_peered":"2024-03-11T19:19:48.249602+0000","last_clean":"2024-03-11T19:19:48.249602+0000","last_became_active":"2024-03-11T19:19:41.397206+0000","last_became_peered":"2024-03-11T19:19:41.397206+0000","last_unstale":"2024-03-11T19:19:48.249602+0000","last_undegraded":"2024-03-11T19:19:48.249602+0000","last_fullsized":"2024-03-11T19:19:48.249602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:24:45.835170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244126+0000","last_change":"2024-03-11T19:19:42.387716+0000","last_active":"2024-03-11T19:19:48.244126+0000","last_peered":"2024-03-11T19:19:48.244126+0000","last_clean":"2024-03-11T19:19:48.244126+0000","last_became_active":"2024-03-11T19:19:42.387370+0000","last_became_peered":"2024-03-11T19:19:42.387370+0000","last_unstale":"2024-03-11T19:19:48.244126+0000","last_undegraded":"2024-03-11T19:19:48.244126+0000","last_fullsized":"2024-03-11T19:19:48.244126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:50:22.852284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792546+0000","last_change":"2024-03-11T19:19:46.827980+0000","last_active":"2024-03-11T19:19:47.792546+0000","last_peered":"2024-03-11T19:19:47.792546+0000","last_clean":"2024-03-11T19:19:47.792546+0000","last_became_active":"2024-03-11T19:19:46.827317+0000","last_became_peered":"2024-03-11T19:19:46.827317+0000","last_unstale":"2024-03-11T19:19:47.792546+0000","last_undegraded":"2024-03-11T19:19:47.792546+0000","last_fullsized":"2024-03-11T19:19:47.792546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:55:15.266791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993451+0000","last_change":"2024-03-11T19:19:41.394714+0000","last_active":"2024-03-11T19:19:47.993451+0000","last_peered":"2024-03-11T19:19:47.993451+0000","last_clean":"2024-03-11T19:19:47.993451+0000","last_became_active":"2024-03-11T19:19:41.394404+0000","last_became_peered":"2024-03-11T19:19:41.394404+0000","last_unstale":"2024-03-11T19:19:47.993451+0000","last_undegraded":"2024-03-11T19:19:47.993451+0000","last_fullsized":"2024-03-11T19:19:47.993451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:15.960037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243905+0000","last_change":"2024-03-11T19:19:42.381062+0000","last_active":"2024-03-11T19:19:48.243905+0000","last_peered":"2024-03-11T19:19:48.243905+0000","last_clean":"2024-03-11T19:19:48.243905+0000","last_became_active":"2024-03-11T19:19:42.380744+0000","last_became_peered":"2024-03-11T19:19:42.380744+0000","last_unstale":"2024-03-11T19:19:48.243905+0000","last_undegraded":"2024-03-11T19:19:48.243905+0000","last_fullsized":"2024-03-11T19:19:48.243905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:08.767877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792802+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:19:47.792802+0000","last_peered":"2024-03-11T19:19:47.792802+0000","last_clean":"2024-03-11T19:19:47.792802+0000","last_became_active":"2024-03-11T19:19:46.934195+0000","last_became_peered":"2024-03-11T19:19:46.934195+0000","last_unstale":"2024-03-11T19:19:47.792802+0000","last_undegraded":"2024-03-11T19:19:47.792802+0000","last_fullsized":"2024-03-11T19:19:47.792802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:33:19.268162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788267+0000","last_change":"2024-03-11T19:19:42.366940+0000","last_active":"2024-03-11T19:19:47.788267+0000","last_peered":"2024-03-11T19:19:47.788267+0000","last_clean":"2024-03-11T19:19:47.788267+0000","last_became_active":"2024-03-11T19:19:42.366628+0000","last_became_peered":"2024-03-11T19:19:42.366628+0000","last_unstale":"2024-03-11T19:19:47.788267+0000","last_undegraded":"2024-03-11T19:19:47.788267+0000","last_fullsized":"2024-03-11T19:19:47.788267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:05:37.175276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.243387+0000","last_change":"2024-03-11T19:19:41.389919+0000","last_active":"2024-03-11T19:20:14.243387+0000","last_peered":"2024-03-11T19:20:14.243387+0000","last_clean":"2024-03-11T19:20:14.243387+0000","last_became_active":"2024-03-11T19:19:41.389471+0000","last_became_peered":"2024-03-11T19:19:41.389471+0000","last_unstale":"2024-03-11T19:20:14.243387+0000","last_undegraded":"2024-03-11T19:20:14.243387+0000","last_fullsized":"2024-03-11T19:20:14.243387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:41.068642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993767+0000","last_change":"2024-03-11T19:19:46.804222+0000","last_active":"2024-03-11T19:19:47.993767+0000","last_peered":"2024-03-11T19:19:47.993767+0000","last_clean":"2024-03-11T19:19:47.993767+0000","last_became_active":"2024-03-11T19:19:46.803756+0000","last_became_peered":"2024-03-11T19:19:46.803756+0000","last_unstale":"2024-03-11T19:19:47.993767+0000","last_undegraded":"2024-03-11T19:19:47.993767+0000","last_fullsized":"2024-03-11T19:19:47.993767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:39.170492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248087+0000","last_change":"2024-03-11T19:19:42.380995+0000","last_active":"2024-03-11T19:19:48.248087+0000","last_peered":"2024-03-11T19:19:48.248087+0000","last_clean":"2024-03-11T19:19:48.248087+0000","last_became_active":"2024-03-11T19:19:42.380672+0000","last_became_peered":"2024-03-11T19:19:42.380672+0000","last_unstale":"2024-03-11T19:19:48.248087+0000","last_undegraded":"2024-03-11T19:19:48.248087+0000","last_fullsized":"2024-03-11T19:19:48.248087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:24:01.289629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.234104+0000","last_change":"2024-03-11T19:19:41.391501+0000","last_active":"2024-03-11T19:20:13.234104+0000","last_peered":"2024-03-11T19:20:13.234104+0000","last_clean":"2024-03-11T19:20:13.234104+0000","last_became_active":"2024-03-11T19:19:41.391148+0000","last_became_peered":"2024-03-11T19:19:41.391148+0000","last_unstale":"2024-03-11T19:20:13.234104+0000","last_undegraded":"2024-03-11T19:20:13.234104+0000","last_fullsized":"2024-03-11T19:20:13.234104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:46:18.957547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993162+0000","last_change":"2024-03-11T19:18:21.278121+0000","last_active":"2024-03-11T19:19:47.993162+0000","last_peered":"2024-03-11T19:19:47.993162+0000","last_clean":"2024-03-11T19:19:47.993162+0000","last_became_active":"2024-03-11T19:18:21.277678+0000","last_became_peered":"2024-03-11T19:18:21.277678+0000","last_unstale":"2024-03-11T19:19:47.993162+0000","last_undegraded":"2024-03-11T19:19:47.993162+0000","last_fullsized":"2024-03-11T19:19:47.993162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_clean_scrub_stamp":"2024-03-11T19:14:20.582658+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:19.162992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788982+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:19:47.788982+0000","last_peered":"2024-03-11T19:19:47.788982+0000","last_clean":"2024-03-11T19:19:47.788982+0000","last_became_active":"2024-03-11T19:19:46.934304+0000","last_became_peered":"2024-03-11T19:19:46.934304+0000","last_unstale":"2024-03-11T19:19:47.788982+0000","last_undegraded":"2024-03-11T19:19:47.788982+0000","last_fullsized":"2024-03-11T19:19:47.788982+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:34:47.392139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.791525+0000","last_change":"2024-03-11T19:19:42.363283+0000","last_active":"2024-03-11T19:19:47.791525+0000","last_peered":"2024-03-11T19:19:47.791525+0000","last_clean":"2024-03-11T19:19:47.791525+0000","last_became_active":"2024-03-11T19:19:42.362930+0000","last_became_peered":"2024-03-11T19:19:42.362930+0000","last_unstale":"2024-03-11T19:19:47.791525+0000","last_undegraded":"2024-03-11T19:19:47.791525+0000","last_fullsized":"2024-03-11T19:19:47.791525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:16:36.141585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.234905+0000","last_change":"2024-03-11T19:19:41.391776+0000","last_active":"2024-03-11T19:20:14.234905+0000","last_peered":"2024-03-11T19:20:14.234905+0000","last_clean":"2024-03-11T19:20:14.234905+0000","last_became_active":"2024-03-11T19:19:41.391459+0000","last_became_peered":"2024-03-11T19:19:41.391459+0000","last_unstale":"2024-03-11T19:20:14.234905+0000","last_undegraded":"2024-03-11T19:20:14.234905+0000","last_fullsized":"2024-03-11T19:20:14.234905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:36.457307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243595+0000","last_change":"2024-03-11T19:19:46.821662+0000","last_active":"2024-03-11T19:19:48.243595+0000","last_peered":"2024-03-11T19:19:48.243595+0000","last_clean":"2024-03-11T19:19:48.243595+0000","last_became_active":"2024-03-11T19:19:46.821199+0000","last_became_peered":"2024-03-11T19:19:46.821199+0000","last_unstale":"2024-03-11T19:19:48.243595+0000","last_undegraded":"2024-03-11T19:19:48.243595+0000","last_fullsized":"2024-03-11T19:19:48.243595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:08:39.658398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244966+0000","last_change":"2024-03-11T19:19:41.399338+0000","last_active":"2024-03-11T19:19:48.244966+0000","last_peered":"2024-03-11T19:19:48.244966+0000","last_clean":"2024-03-11T19:19:48.244966+0000","last_became_active":"2024-03-11T19:19:41.399027+0000","last_became_peered":"2024-03-11T19:19:41.399027+0000","last_unstale":"2024-03-11T19:19:48.244966+0000","last_undegraded":"2024-03-11T19:19:48.244966+0000","last_fullsized":"2024-03-11T19:19:48.244966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:26:39.157631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792816+0000","last_change":"2024-03-11T19:19:42.380304+0000","last_active":"2024-03-11T19:19:47.792816+0000","last_peered":"2024-03-11T19:19:47.792816+0000","last_clean":"2024-03-11T19:19:47.792816+0000","last_became_active":"2024-03-11T19:19:42.378768+0000","last_became_peered":"2024-03-11T19:19:42.378768+0000","last_unstale":"2024-03-11T19:19:47.792816+0000","last_undegraded":"2024-03-11T19:19:47.792816+0000","last_fullsized":"2024-03-11T19:19:47.792816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:31.465876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246325+0000","last_change":"2024-03-11T19:19:46.934262+0000","last_active":"2024-03-11T19:19:48.246325+0000","last_peered":"2024-03-11T19:19:48.246325+0000","last_clean":"2024-03-11T19:19:48.246325+0000","last_became_active":"2024-03-11T19:19:46.933557+0000","last_became_peered":"2024-03-11T19:19:46.933557+0000","last_unstale":"2024-03-11T19:19:48.246325+0000","last_undegraded":"2024-03-11T19:19:48.246325+0000","last_fullsized":"2024-03-11T19:19:48.246325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:29.227551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:14.237308+0000","last_change":"2024-03-11T19:19:41.402974+0000","last_active":"2024-03-11T19:20:14.237308+0000","last_peered":"2024-03-11T19:20:14.237308+0000","last_clean":"2024-03-11T19:20:14.237308+0000","last_became_active":"2024-03-11T19:19:41.402257+0000","last_became_peered":"2024-03-11T19:19:41.402257+0000","last_unstale":"2024-03-11T19:20:14.237308+0000","last_undegraded":"2024-03-11T19:20:14.237308+0000","last_fullsized":"2024-03-11T19:20:14.237308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:00:27.136592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250098+0000","last_change":"2024-03-11T19:19:42.385331+0000","last_active":"2024-03-11T19:19:48.250098+0000","last_peered":"2024-03-11T19:19:48.250098+0000","last_clean":"2024-03-11T19:19:48.250098+0000","last_became_active":"2024-03-11T19:19:42.381750+0000","last_became_peered":"2024-03-11T19:19:42.381750+0000","last_unstale":"2024-03-11T19:19:48.250098+0000","last_undegraded":"2024-03-11T19:19:48.250098+0000","last_fullsized":"2024-03-11T19:19:48.250098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:54:20.851516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243200+0000","last_change":"2024-03-11T19:19:46.801276+0000","last_active":"2024-03-11T19:19:48.243200+0000","last_peered":"2024-03-11T19:19:48.243200+0000","last_clean":"2024-03-11T19:19:48.243200+0000","last_became_active":"2024-03-11T19:19:46.800791+0000","last_became_peered":"2024-03-11T19:19:46.800791+0000","last_unstale":"2024-03-11T19:19:48.243200+0000","last_undegraded":"2024-03-11T19:19:48.243200+0000","last_fullsized":"2024-03-11T19:19:48.243200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:05.418175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.231473+0000","last_change":"2024-03-11T19:19:41.392788+0000","last_active":"2024-03-11T19:20:13.231473+0000","last_peered":"2024-03-11T19:20:13.231473+0000","last_clean":"2024-03-11T19:20:13.231473+0000","last_became_active":"2024-03-11T19:19:41.392483+0000","last_became_peered":"2024-03-11T19:19:41.392483+0000","last_unstale":"2024-03-11T19:20:13.231473+0000","last_undegraded":"2024-03-11T19:20:13.231473+0000","last_fullsized":"2024-03-11T19:20:13.231473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:55.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.246121+0000","last_change":"2024-03-11T19:19:42.394123+0000","last_active":"2024-03-11T19:19:48.246121+0000","last_peered":"2024-03-11T19:19:48.246121+0000","last_clean":"2024-03-11T19:19:48.246121+0000","last_became_active":"2024-03-11T19:19:42.393785+0000","last_became_peered":"2024-03-11T19:19:42.393785+0000","last_unstale":"2024-03-11T19:19:48.246121+0000","last_undegraded":"2024-03-11T19:19:48.246121+0000","last_fullsized":"2024-03-11T19:19:48.246121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:46:24.299553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243536+0000","last_change":"2024-03-11T19:19:46.934434+0000","last_active":"2024-03-11T19:19:48.243536+0000","last_peered":"2024-03-11T19:19:48.243536+0000","last_clean":"2024-03-11T19:19:48.243536+0000","last_became_active":"2024-03-11T19:19:46.933925+0000","last_became_peered":"2024-03-11T19:19:46.933925+0000","last_unstale":"2024-03-11T19:19:48.243536+0000","last_undegraded":"2024-03-11T19:19:48.243536+0000","last_fullsized":"2024-03-11T19:19:48.243536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:33:09.098595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.231284+0000","last_change":"2024-03-11T19:19:41.394099+0000","last_active":"2024-03-11T19:20:13.231284+0000","last_peered":"2024-03-11T19:20:13.231284+0000","last_clean":"2024-03-11T19:20:13.231284+0000","last_became_active":"2024-03-11T19:19:41.393780+0000","last_became_peered":"2024-03-11T19:19:41.393780+0000","last_unstale":"2024-03-11T19:20:13.231284+0000","last_undegraded":"2024-03-11T19:20:13.231284+0000","last_fullsized":"2024-03-11T19:20:13.231284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:38:16.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243683+0000","last_change":"2024-03-11T19:19:42.363963+0000","last_active":"2024-03-11T19:19:48.243683+0000","last_peered":"2024-03-11T19:19:48.243683+0000","last_clean":"2024-03-11T19:19:48.243683+0000","last_became_active":"2024-03-11T19:19:42.363315+0000","last_became_peered":"2024-03-11T19:19:42.363315+0000","last_unstale":"2024-03-11T19:19:48.243683+0000","last_undegraded":"2024-03-11T19:19:48.243683+0000","last_fullsized":"2024-03-11T19:19:48.243683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:13:14.828230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250483+0000","last_change":"2024-03-11T19:19:46.808879+0000","last_active":"2024-03-11T19:19:48.250483+0000","last_peered":"2024-03-11T19:19:48.250483+0000","last_clean":"2024-03-11T19:19:48.250483+0000","last_became_active":"2024-03-11T19:19:46.808428+0000","last_became_peered":"2024-03-11T19:19:46.808428+0000","last_unstale":"2024-03-11T19:19:48.250483+0000","last_undegraded":"2024-03-11T19:19:48.250483+0000","last_fullsized":"2024-03-11T19:19:48.250483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:19.024854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.234510+0000","last_change":"2024-03-11T19:19:41.395487+0000","last_active":"2024-03-11T19:20:13.234510+0000","last_peered":"2024-03-11T19:20:13.234510+0000","last_clean":"2024-03-11T19:20:13.234510+0000","last_became_active":"2024-03-11T19:19:41.395081+0000","last_became_peered":"2024-03-11T19:19:41.395081+0000","last_unstale":"2024-03-11T19:20:13.234510+0000","last_undegraded":"2024-03-11T19:20:13.234510+0000","last_fullsized":"2024-03-11T19:20:13.234510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:54:58.855745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243302+0000","last_change":"2024-03-11T19:19:42.378860+0000","last_active":"2024-03-11T19:19:48.243302+0000","last_peered":"2024-03-11T19:19:48.243302+0000","last_clean":"2024-03-11T19:19:48.243302+0000","last_became_active":"2024-03-11T19:19:42.378421+0000","last_became_peered":"2024-03-11T19:19:42.378421+0000","last_unstale":"2024-03-11T19:19:48.243302+0000","last_undegraded":"2024-03-11T19:19:48.243302+0000","last_fullsized":"2024-03-11T19:19:48.243302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:08.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243115+0000","last_change":"2024-03-11T19:19:46.815770+0000","last_active":"2024-03-11T19:19:48.243115+0000","last_peered":"2024-03-11T19:19:48.243115+0000","last_clean":"2024-03-11T19:19:48.243115+0000","last_became_active":"2024-03-11T19:19:46.814982+0000","last_became_peered":"2024-03-11T19:19:46.814982+0000","last_unstale":"2024-03-11T19:19:48.243115+0000","last_undegraded":"2024-03-11T19:19:48.243115+0000","last_fullsized":"2024-03-11T19:19:48.243115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:19.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243744+0000","last_change":"2024-03-11T19:19:41.397753+0000","last_active":"2024-03-11T19:19:48.243744+0000","last_peered":"2024-03-11T19:19:48.243744+0000","last_clean":"2024-03-11T19:19:48.243744+0000","last_became_active":"2024-03-11T19:19:41.397200+0000","last_became_peered":"2024-03-11T19:19:41.397200+0000","last_unstale":"2024-03-11T19:19:48.243744+0000","last_undegraded":"2024-03-11T19:19:48.243744+0000","last_fullsized":"2024-03-11T19:19:48.243744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:38:38.326990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248865+0000","last_change":"2024-03-11T19:19:42.387285+0000","last_active":"2024-03-11T19:19:48.248865+0000","last_peered":"2024-03-11T19:19:48.248865+0000","last_clean":"2024-03-11T19:19:48.248865+0000","last_became_active":"2024-03-11T19:19:42.386919+0000","last_became_peered":"2024-03-11T19:19:42.386919+0000","last_unstale":"2024-03-11T19:19:48.248865+0000","last_undegraded":"2024-03-11T19:19:48.248865+0000","last_fullsized":"2024-03-11T19:19:48.248865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.773177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243817+0000","last_change":"2024-03-11T19:19:46.934177+0000","last_active":"2024-03-11T19:19:48.243817+0000","last_peered":"2024-03-11T19:19:48.243817+0000","last_clean":"2024-03-11T19:19:48.243817+0000","last_became_active":"2024-03-11T19:19:46.933699+0000","last_became_peered":"2024-03-11T19:19:46.933699+0000","last_unstale":"2024-03-11T19:19:48.243817+0000","last_undegraded":"2024-03-11T19:19:48.243817+0000","last_fullsized":"2024-03-11T19:19:48.243817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:09:17.433172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:20:13.230989+0000","last_change":"2024-03-11T19:19:41.396888+0000","last_active":"2024-03-11T19:20:13.230989+0000","last_peered":"2024-03-11T19:20:13.230989+0000","last_clean":"2024-03-11T19:20:13.230989+0000","last_became_active":"2024-03-11T19:19:41.396640+0000","last_became_peered":"2024-03-11T19:19:41.396640+0000","last_unstale":"2024-03-11T19:20:13.230989+0000","last_undegraded":"2024-03-11T19:20:13.230989+0000","last_fullsized":"2024-03-11T19:20:13.230989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:00.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993490+0000","last_change":"2024-03-11T19:19:42.370020+0000","last_active":"2024-03-11T19:19:47.993490+0000","last_peered":"2024-03-11T19:19:47.993490+0000","last_clean":"2024-03-11T19:19:47.993490+0000","last_became_active":"2024-03-11T19:19:42.369355+0000","last_became_peered":"2024-03-11T19:19:42.369355+0000","last_unstale":"2024-03-11T19:19:47.993490+0000","last_undegraded":"2024-03-11T19:19:47.993490+0000","last_fullsized":"2024-03-11T19:19:47.993490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:07:13.572419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788219+0000","last_change":"2024-03-11T19:19:46.818424+0000","last_active":"2024-03-11T19:19:47.788219+0000","last_peered":"2024-03-11T19:19:47.788219+0000","last_clean":"2024-03-11T19:19:47.788219+0000","last_became_active":"2024-03-11T19:19:46.817934+0000","last_became_peered":"2024-03-11T19:19:46.817934+0000","last_unstale":"2024-03-11T19:19:47.788219+0000","last_undegraded":"2024-03-11T19:19:47.788219+0000","last_fullsized":"2024-03-11T19:19:47.788219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:52:56.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244742+0000","last_change":"2024-03-11T19:19:41.399466+0000","last_active":"2024-03-11T19:19:48.244742+0000","last_peered":"2024-03-11T19:19:48.244742+0000","last_clean":"2024-03-11T19:19:48.244742+0000","last_became_active":"2024-03-11T19:19:41.399118+0000","last_became_peered":"2024-03-11T19:19:41.399118+0000","last_unstale":"2024-03-11T19:19:48.244742+0000","last_undegraded":"2024-03-11T19:19:48.244742+0000","last_fullsized":"2024-03-11T19:19:48.244742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:12:46.100596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993861+0000","last_change":"2024-03-11T19:19:42.365437+0000","last_active":"2024-03-11T19:19:47.993861+0000","last_peered":"2024-03-11T19:19:47.993861+0000","last_clean":"2024-03-11T19:19:47.993861+0000","last_became_active":"2024-03-11T19:19:42.365084+0000","last_became_peered":"2024-03-11T19:19:42.365084+0000","last_unstale":"2024-03-11T19:19:47.993861+0000","last_undegraded":"2024-03-11T19:19:47.993861+0000","last_fullsized":"2024-03-11T19:19:47.993861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:35.239100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245487+0000","last_change":"2024-03-11T19:19:46.825067+0000","last_active":"2024-03-11T19:19:48.245487+0000","last_peered":"2024-03-11T19:19:48.245487+0000","last_clean":"2024-03-11T19:19:48.245487+0000","last_became_active":"2024-03-11T19:19:46.821438+0000","last_became_peered":"2024-03-11T19:19:46.821438+0000","last_unstale":"2024-03-11T19:19:48.245487+0000","last_undegraded":"2024-03-11T19:19:48.245487+0000","last_fullsized":"2024-03-11T19:19:48.245487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:08.828669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.789022+0000","last_change":"2024-03-11T19:19:41.403788+0000","last_active":"2024-03-11T19:19:47.789022+0000","last_peered":"2024-03-11T19:19:47.789022+0000","last_clean":"2024-03-11T19:19:47.789022+0000","last_became_active":"2024-03-11T19:19:41.403479+0000","last_became_peered":"2024-03-11T19:19:41.403479+0000","last_unstale":"2024-03-11T19:19:47.789022+0000","last_undegraded":"2024-03-11T19:19:47.789022+0000","last_fullsized":"2024-03-11T19:19:47.789022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:58:03.407100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993519+0000","last_change":"2024-03-11T19:19:42.381891+0000","last_active":"2024-03-11T19:19:47.993519+0000","last_peered":"2024-03-11T19:19:47.993519+0000","last_clean":"2024-03-11T19:19:47.993519+0000","last_became_active":"2024-03-11T19:19:42.381573+0000","last_became_peered":"2024-03-11T19:19:42.381573+0000","last_unstale":"2024-03-11T19:19:47.993519+0000","last_undegraded":"2024-03-11T19:19:47.993519+0000","last_fullsized":"2024-03-11T19:19:47.993519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:25.967035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788200+0000","last_change":"2024-03-11T19:19:46.803241+0000","last_active":"2024-03-11T19:19:47.788200+0000","last_peered":"2024-03-11T19:19:47.788200+0000","last_clean":"2024-03-11T19:19:47.788200+0000","last_became_active":"2024-03-11T19:19:46.802774+0000","last_became_peered":"2024-03-11T19:19:46.802774+0000","last_unstale":"2024-03-11T19:19:47.788200+0000","last_undegraded":"2024-03-11T19:19:47.788200+0000","last_fullsized":"2024-03-11T19:19:47.788200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:12.164870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792566+0000","last_change":"2024-03-11T19:19:41.394613+0000","last_active":"2024-03-11T19:19:47.792566+0000","last_peered":"2024-03-11T19:19:47.792566+0000","last_clean":"2024-03-11T19:19:47.792566+0000","last_became_active":"2024-03-11T19:19:41.394252+0000","last_became_peered":"2024-03-11T19:19:41.394252+0000","last_unstale":"2024-03-11T19:19:47.792566+0000","last_undegraded":"2024-03-11T19:19:47.792566+0000","last_fullsized":"2024-03-11T19:19:47.792566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:05:50.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.993829+0000","last_change":"2024-03-11T19:19:42.366706+0000","last_active":"2024-03-11T19:19:47.993829+0000","last_peered":"2024-03-11T19:19:47.993829+0000","last_clean":"2024-03-11T19:19:47.993829+0000","last_became_active":"2024-03-11T19:19:42.366389+0000","last_became_peered":"2024-03-11T19:19:42.366389+0000","last_unstale":"2024-03-11T19:19:47.993829+0000","last_undegraded":"2024-03-11T19:19:47.993829+0000","last_fullsized":"2024-03-11T19:19:47.993829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:15:08.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243425+0000","last_change":"2024-03-11T19:19:42.367886+0000","last_active":"2024-03-11T19:19:48.243425+0000","last_peered":"2024-03-11T19:19:48.243425+0000","last_clean":"2024-03-11T19:19:48.243425+0000","last_became_active":"2024-03-11T19:19:42.367543+0000","last_became_peered":"2024-03-11T19:19:42.367543+0000","last_unstale":"2024-03-11T19:19:48.243425+0000","last_undegraded":"2024-03-11T19:19:48.243425+0000","last_fullsized":"2024-03-11T19:19:48.243425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:15:01.606205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243826+0000","last_change":"2024-03-11T19:19:42.370189+0000","last_active":"2024-03-11T19:19:48.243826+0000","last_peered":"2024-03-11T19:19:48.243826+0000","last_clean":"2024-03-11T19:19:48.243826+0000","last_became_active":"2024-03-11T19:19:42.369553+0000","last_became_peered":"2024-03-11T19:19:42.369553+0000","last_unstale":"2024-03-11T19:19:48.243826+0000","last_undegraded":"2024-03-11T19:19:48.243826+0000","last_fullsized":"2024-03-11T19:19:48.243826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:10:09.973215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243617+0000","last_change":"2024-03-11T19:19:42.380853+0000","last_active":"2024-03-11T19:19:48.243617+0000","last_peered":"2024-03-11T19:19:48.243617+0000","last_clean":"2024-03-11T19:19:48.243617+0000","last_became_active":"2024-03-11T19:19:42.380502+0000","last_became_peered":"2024-03-11T19:19:42.380502+0000","last_unstale":"2024-03-11T19:19:48.243617+0000","last_undegraded":"2024-03-11T19:19:48.243617+0000","last_fullsized":"2024-03-11T19:19:48.243617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:22:20.832801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.245513+0000","last_change":"2024-03-11T19:19:42.382850+0000","last_active":"2024-03-11T19:19:48.245513+0000","last_peered":"2024-03-11T19:19:48.245513+0000","last_clean":"2024-03-11T19:19:48.245513+0000","last_became_active":"2024-03-11T19:19:42.382134+0000","last_became_peered":"2024-03-11T19:19:42.382134+0000","last_unstale":"2024-03-11T19:19:48.245513+0000","last_undegraded":"2024-03-11T19:19:48.245513+0000","last_fullsized":"2024-03-11T19:19:48.245513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:49.859290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.792951+0000","last_change":"2024-03-11T19:19:42.386882+0000","last_active":"2024-03-11T19:19:47.792951+0000","last_peered":"2024-03-11T19:19:47.792951+0000","last_clean":"2024-03-11T19:19:47.792951+0000","last_became_active":"2024-03-11T19:19:42.386280+0000","last_became_peered":"2024-03-11T19:19:42.386280+0000","last_unstale":"2024-03-11T19:19:47.792951+0000","last_undegraded":"2024-03-11T19:19:47.792951+0000","last_fullsized":"2024-03-11T19:19:47.792951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:30:10.891093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244001+0000","last_change":"2024-03-11T19:19:42.368025+0000","last_active":"2024-03-11T19:19:48.244001+0000","last_peered":"2024-03-11T19:19:48.244001+0000","last_clean":"2024-03-11T19:19:48.244001+0000","last_became_active":"2024-03-11T19:19:42.367695+0000","last_became_peered":"2024-03-11T19:19:42.367695+0000","last_unstale":"2024-03-11T19:19:48.244001+0000","last_undegraded":"2024-03-11T19:19:48.244001+0000","last_fullsized":"2024-03-11T19:19:48.244001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:11:44.776358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.244842+0000","last_change":"2024-03-11T19:19:42.364940+0000","last_active":"2024-03-11T19:19:48.244842+0000","last_peered":"2024-03-11T19:19:48.244842+0000","last_clean":"2024-03-11T19:19:48.244842+0000","last_became_active":"2024-03-11T19:19:42.364620+0000","last_became_peered":"2024-03-11T19:19:42.364620+0000","last_unstale":"2024-03-11T19:19:48.244842+0000","last_undegraded":"2024-03-11T19:19:48.244842+0000","last_fullsized":"2024-03-11T19:19:48.244842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:11.415716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.250406+0000","last_change":"2024-03-11T19:19:42.392468+0000","last_active":"2024-03-11T19:19:48.250406+0000","last_peered":"2024-03-11T19:19:48.250406+0000","last_clean":"2024-03-11T19:19:48.250406+0000","last_became_active":"2024-03-11T19:19:42.390521+0000","last_became_peered":"2024-03-11T19:19:42.390521+0000","last_unstale":"2024-03-11T19:19:48.250406+0000","last_undegraded":"2024-03-11T19:19:48.250406+0000","last_fullsized":"2024-03-11T19:19:48.250406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:51:21.487796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788259+0000","last_change":"2024-03-11T19:19:42.393130+0000","last_active":"2024-03-11T19:19:47.788259+0000","last_peered":"2024-03-11T19:19:47.788259+0000","last_clean":"2024-03-11T19:19:47.788259+0000","last_became_active":"2024-03-11T19:19:42.392845+0000","last_became_peered":"2024-03-11T19:19:42.392845+0000","last_unstale":"2024-03-11T19:19:47.788259+0000","last_undegraded":"2024-03-11T19:19:47.788259+0000","last_fullsized":"2024-03-11T19:19:47.788259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:17:27.278574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.248711+0000","last_change":"2024-03-11T19:19:42.385890+0000","last_active":"2024-03-11T19:19:48.248711+0000","last_peered":"2024-03-11T19:19:48.248711+0000","last_clean":"2024-03-11T19:19:48.248711+0000","last_became_active":"2024-03-11T19:19:42.381939+0000","last_became_peered":"2024-03-11T19:19:42.381939+0000","last_unstale":"2024-03-11T19:19:48.248711+0000","last_undegraded":"2024-03-11T19:19:48.248711+0000","last_fullsized":"2024-03-11T19:19:48.248711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:47:11.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.788317+0000","last_change":"2024-03-11T19:19:42.392856+0000","last_active":"2024-03-11T19:19:47.788317+0000","last_peered":"2024-03-11T19:19:47.788317+0000","last_clean":"2024-03-11T19:19:47.788317+0000","last_became_active":"2024-03-11T19:19:42.392554+0000","last_became_peered":"2024-03-11T19:19:42.392554+0000","last_unstale":"2024-03-11T19:19:47.788317+0000","last_undegraded":"2024-03-11T19:19:47.788317+0000","last_fullsized":"2024-03-11T19:19:47.788317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:54:04.840145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:47.791946+0000","last_change":"2024-03-11T19:19:42.383082+0000","last_active":"2024-03-11T19:19:47.791946+0000","last_peered":"2024-03-11T19:19:47.791946+0000","last_clean":"2024-03-11T19:19:47.791946+0000","last_became_active":"2024-03-11T19:19:42.381546+0000","last_became_peered":"2024-03-11T19:19:42.381546+0000","last_unstale":"2024-03-11T19:19:47.791946+0000","last_undegraded":"2024-03-11T19:19:47.791946+0000","last_fullsized":"2024-03-11T19:19:47.791946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:20.493148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:19:48.243964+0000","last_change":"2024-03-11T19:19:42.369136+0000","last_active":"2024-03-11T19:19:48.243964+0000","last_peered":"2024-03-11T19:19:48.243964+0000","last_clean":"2024-03-11T19:19:48.243964+0000","last_became_active":"2024-03-11T19:19:42.368807+0000","last_became_peered":"2024-03-11T19:19:42.368807+0000","last_unstale":"2024-03-11T19:19:48.243964+0000","last_undegraded":"2024-03-11T19:19:48.243964+0000","last_fullsized":"2024-03-11T19:19:48.243964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:16:34.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281180,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43499,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":14,"num_read_kb":48,"num_write":133,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":58491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":480845,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1100954,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481663,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":481471,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1081588,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480935,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484687,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482323,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485343,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481129,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":481489,"data_compressed":14670,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1072689,"data_compressed":50434,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:20:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-11T19:20:19.524 INFO:tasks.cephadm:Setting up client nodes... 2024-03-11T19:20:19.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-11T19:20:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:19 smithi106 ceph-mon[35889]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:19 smithi106 ceph-mon[35889]: pgmap v344: 129 pgs: 129 active+clean; 619 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-03-11T19:20:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:19 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/559960991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:19.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:19 smithi143 ceph-mon[36013]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:20:19.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:19 smithi143 ceph-mon[36013]: pgmap v344: 129 pgs: 129 active+clean; 619 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-03-11T19:20:19.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:19 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/559960991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:19.995 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:20.429 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:20 smithi072 ceph-mon[31695]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:20.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:20 smithi106 ceph-mon[35889]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:20.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:20 smithi143 ceph-mon[36013]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:21.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:21 smithi072 ceph-mon[31695]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:21.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:21 smithi072 ceph-mon[31695]: pgmap v345: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:20:21.546 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:21 smithi106 ceph-mon[35889]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:21.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:21 smithi106 ceph-mon[35889]: pgmap v345: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:20:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:21 smithi143 ceph-mon[36013]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:20:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:21 smithi143 ceph-mon[36013]: pgmap v345: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 -- 172.21.15.72:0/1603026946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 msgr2=0x7fb61410c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 --2- 172.21.15.72:0/1603026946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb61410c370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b0a0 tx=0x7fb60c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 -- 172.21.15.72:0/1603026946 shutdown_connections 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 --2- 172.21.15.72:0/1603026946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb61410c370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 --2- 172.21.15.72:0/1603026946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614103ea0 0x7fb6141042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 --2- 172.21.15.72:0/1603026946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614105090 0x7fb614105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 -- 172.21.15.72:0/1603026946 >> 172.21.15.72:0/1603026946 conn(0x7fb6140ff490 msgr2=0x7fb6141018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:22.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 -- 172.21.15.72:0/1603026946 shutdown_connections 2024-03-11T19:20:22.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.600+0000 7fb61b600700 1 -- 172.21.15.72:0/1603026946 wait complete. 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.601+0000 7fb61b600700 1 Processor -- start 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.601+0000 7fb61b600700 1 -- start start 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614105090 0x7fb61419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb6141a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6141a1a40 con 0x7fb614105a50 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6141a1b80 con 0x7fb614103ea0 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb61b600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6141a1d10 con 0x7fb614105090 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb619600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb6141a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb618c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb613e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614105090 0x7fb61419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb618c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:48798/0 (socket says 172.21.15.72:48798) 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.602+0000 7fb618c00700 1 -- 172.21.15.72:0/1692895530 learned_addr learned my addr 172.21.15.72:0/1692895530 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 -- 172.21.15.72:0/1692895530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614105090 msgr2=0x7fb61419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614105090 0x7fb61419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 -- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 msgr2=0x7fb6141a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb6141a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 -- 172.21.15.72:0/1692895530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c009d00 con 0x7fb614103ea0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb618c00700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb60800e970 tx=0x7fb60800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608009790 con 0x7fb614103ea0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb60800ccc0 con 0x7fb614103ea0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6141a1ff0 con 0x7fb614103ea0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608003d60 con 0x7fb614103ea0 2024-03-11T19:20:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.603+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6141a2410 con 0x7fb614103ea0 2024-03-11T19:20:22.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.604+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb614050180 con 0x7fb614103ea0 2024-03-11T19:20:22.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.607+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fb60800c7e0 con 0x7fb614103ea0 2024-03-11T19:20:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.608+0000 7fb611600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 0x7fb5fc0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.608+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb608014070 con 0x7fb614103ea0 2024-03-11T19:20:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.608+0000 7fb613e00700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 0x7fb5fc0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:22.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.609+0000 7fb613e00700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 0x7fb5fc0a4140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb614103bd0 tx=0x7fb600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:22.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.610+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb608096900 con 0x7fb614103ea0 2024-03-11T19:20:22.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.913+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb61410b210 con 0x7fb614103ea0 2024-03-11T19:20:22.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.922+0000 7fb611600700 1 -- 172.21.15.72:0/1692895530 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb608099fc0 con 0x7fb614103ea0 2024-03-11T19:20:22.926 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2024-03-11T19:20:22.926 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQB2We9loTifNhAANWUSGzizgFJQY+R1S1S8Lg== 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 msgr2=0x7fb5fc0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 0x7fb5fc0a4140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb614103bd0 tx=0x7fb600009260 comp rx=0 tx=0).stop 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 msgr2=0x7fb614199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb60800e970 tx=0x7fb60800ec80 comp rx=0 tx=0).stop 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 shutdown_connections 2024-03-11T19:20:22.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5fc0a1c80 0x7fb5fc0a4140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb614105a50 0x7fb6141a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb614103ea0 0x7fb614199eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 --2- 172.21.15.72:0/1692895530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb614105090 0x7fb61419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 >> 172.21.15.72:0/1692895530 conn(0x7fb6140ff490 msgr2=0x7fb614108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 shutdown_connections 2024-03-11T19:20:22.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:22.925+0000 7fb61b600700 1 -- 172.21.15.72:0/1692895530 wait complete. 2024-03-11T19:20:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:23 smithi106 ceph-mon[35889]: pgmap v346: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-03-11T19:20:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:23 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1692895530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:23 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:23 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:20:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:23 smithi072 ceph-mon[31695]: pgmap v346: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-03-11T19:20:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:23 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1692895530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:23 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:23 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:20:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:23 smithi143 ceph-mon[36013]: pgmap v346: 129 pgs: 129 active+clean; 620 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-03-11T19:20:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:23 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1692895530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:23 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:20:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:23 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:20:23.873 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:20:23.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-11T19:20:23.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-11T19:20:24.028 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-11T19:20:24.029 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-11T19:20:24.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-11T19:20:24.512 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:25.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:25 smithi106 ceph-mon[35889]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-11T19:20:25.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:25 smithi072 ceph-mon[31695]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-11T19:20:25.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:25 smithi143 ceph-mon[36013]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-11T19:20:27.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 -- 172.21.15.72:0/3981644865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 msgr2=0x7f4ec406ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 --2- 172.21.15.72:0/3981644865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec406ab70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009990 tx=0x7f4eb0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 -- 172.21.15.72:0/3981644865 shutdown_connections 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 --2- 172.21.15.72:0/3981644865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec41046a0 0x7f4ec4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 --2- 172.21.15.72:0/3981644865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec4106c30 0x7f4ec406a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 --2- 172.21.15.72:0/3981644865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec406ab70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 -- 172.21.15.72:0/3981644865 >> 172.21.15.72:0/3981644865 conn(0x7f4ec4076bc0 msgr2=0x7f4ec4076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 -- 172.21.15.72:0/3981644865 shutdown_connections 2024-03-11T19:20:27.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.287+0000 7f4ec9800700 1 -- 172.21.15.72:0/3981644865 wait complete. 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 Processor -- start 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 -- start start 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec4106c30 0x7f4ec41a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a35a0 con 0x7f4ec406a6f0 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a3710 con 0x7f4ec4106c30 2024-03-11T19:20:27.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.289+0000 7f4ec9800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a38d0 con 0x7f4ec41046a0 2024-03-11T19:20:27.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:27.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:51736/0 (socket says 172.21.15.72:51736) 2024-03-11T19:20:27.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 -- 172.21.15.72:0/1761498314 learned_addr learned my addr 172.21.15.72:0/1761498314 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec3400700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec4106c30 0x7f4ec41a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2a00700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 -- 172.21.15.72:0/1761498314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec4106c30 msgr2=0x7f4ec41a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec4106c30 0x7f4ec41a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 -- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 msgr2=0x7f4ec419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2000700 1 -- 172.21.15.72:0/1761498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc009c40 con 0x7f4ec41046a0 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.290+0000 7f4ec2a00700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:20:27.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ec2000700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005910 tx=0x7f4eb0009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001d070 con 0x7f4ec41046a0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eb000b5b0 con 0x7f4ec41046a0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00173f0 con 0x7f4ec41046a0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb00095f0 con 0x7f4ec41046a0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.291+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec41a3de0 con 0x7f4ec41046a0 2024-03-11T19:20:27.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.292+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec4068930 con 0x7f4ec41046a0 2024-03-11T19:20:27.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.294+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4eb0022470 con 0x7f4ec41046a0 2024-03-11T19:20:27.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.295+0000 7f4ebb400700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 0x7f4ea80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:27.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.296+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4eb00cea80 con 0x7f4ec41046a0 2024-03-11T19:20:27.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.296+0000 7f4ec2a00700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 0x7f4ea80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:27.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.296+0000 7f4ec2a00700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 0x7f4ea80a4090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009cb0 tx=0x7f4eb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:27.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.300+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4eb0097920 con 0x7f4ec41046a0 2024-03-11T19:20:27.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:27 smithi072 ceph-mon[31695]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-11T19:20:27.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.584+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4ec41182f0 con 0x7f4ec41046a0 2024-03-11T19:20:27.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.590+0000 7f4ebb400700 1 -- 172.21.15.72:0/1761498314 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260659 (secure 0 0 0) 0x7f4eb009afe0 con 0x7f4ec41046a0 2024-03-11T19:20:27.600 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 msgr2=0x7f4ea80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 0x7f4ea80a4090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009cb0 tx=0x7f4eb4009260 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 msgr2=0x7f4ec419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005910 tx=0x7f4eb0009570 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 shutdown_connections 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ea80a1bd0 0x7f4ea80a4090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec4106c30 0x7f4ec41a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ec406a6f0 0x7f4ec419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 --2- 172.21.15.72:0/1761498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ec41046a0 0x7f4ec419ec00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 >> 172.21.15.72:0/1761498314 conn(0x7f4ec4076bc0 msgr2=0x7f4ec40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 shutdown_connections 2024-03-11T19:20:27.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:27.598+0000 7f4ec9800700 1 -- 172.21.15.72:0/1761498314 wait complete. 2024-03-11T19:20:27.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:27 smithi106 ceph-mon[35889]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-11T19:20:27.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:27 smithi143 ceph-mon[36013]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-11T19:20:28.505 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":537254334},{"type":"v1","addr":"172.21.15.72:6801","nonce":537254334}]},"active_addr":"172.21.15.72:6801/537254334","active_change":"2024-03-11T19:11:29.910077+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.72:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":3915050930}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":828846469}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2158720129}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":4059778386}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1965444324}]}]} 2024-03-11T19:20:28.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-11T19:20:28.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-11T19:20:28.511 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:20:28.625 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:28 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1761498314' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:20:28.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:28 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1761498314' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:20:28.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:28 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1761498314' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:20:28.979 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:29.408 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:29 smithi072 ceph-mon[31695]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-11T19:20:29.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:29 smithi106 ceph-mon[35889]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-11T19:20:29.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:29 smithi143 ceph-mon[36013]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-11T19:20:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-11T19:20:30.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:30 smithi072 ceph-mon[31695]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-11T19:20:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-11T19:20:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:20:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-03-11T19:20:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-11T19:20:30.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:30 smithi106 ceph-mon[35889]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-11T19:20:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-11T19:20:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:20:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-03-11T19:20:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-11T19:20:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-11T19:20:30.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:30 smithi143 ceph-mon[36013]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-11T19:20:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-11T19:20:31.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:31 smithi072 ceph-mon[31695]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:20:31.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 -- 172.21.15.72:0/2283349104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb4108e90 msgr2=0x7fdcb410cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:31.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 --2- 172.21.15.72:0/2283349104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb4108e90 0x7fdcb410cee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009990 tx=0x7fdca0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 -- 172.21.15.72:0/2283349104 shutdown_connections 2024-03-11T19:20:31.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 --2- 172.21.15.72:0/2283349104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 0x7fdcb4115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 --2- 172.21.15.72:0/2283349104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb41084e0 0x7fdcb41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 --2- 172.21.15.72:0/2283349104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb4108e90 0x7fdcb410cee0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 -- 172.21.15.72:0/2283349104 >> 172.21.15.72:0/2283349104 conn(0x7fdcb4075440 msgr2=0x7fdcb4077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 -- 172.21.15.72:0/2283349104 shutdown_connections 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.464+0000 7fdcbc400700 1 -- 172.21.15.72:0/2283349104 wait complete. 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.465+0000 7fdcbc400700 1 Processor -- start 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.465+0000 7fdcbc400700 1 -- start start 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb4108e90 0x7fdcb41a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 0x7fdcb419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419ec40 con 0x7fdcb41084e0 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419ed80 con 0x7fdcb4108e90 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcbc400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb419ef40 con 0x7fdcb410d5c0 2024-03-11T19:20:31.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57518/0 (socket says 172.21.15.72:57518) 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 -- 172.21.15.72:0/1837402500 learned_addr learned my addr 172.21.15.72:0/1837402500 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9000700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb4108e90 0x7fdcb41a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcba400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 0x7fdcb419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 -- 172.21.15.72:0/1837402500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 msgr2=0x7fdcb419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 0x7fdcb419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 -- 172.21.15.72:0/1837402500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb4108e90 msgr2=0x7fdcb41a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb4108e90 0x7fdcb41a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.466+0000 7fdcb9a00700 1 -- 172.21.15.72:0/1837402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00095f0 con 0x7fdcb41084e0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcb9a00700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdca800d980 tx=0x7fdca800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8004220 con 0x7fdcb41084e0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdca8007640 con 0x7fdcb41084e0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb419f220 con 0x7fdcb41084e0 2024-03-11T19:20:31.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca800f690 con 0x7fdcb41084e0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.467+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41ab4c0 con 0x7fdcb41084e0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.468+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb41108e0 con 0x7fdcb41084e0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.470+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fdca8004410 con 0x7fdcb41084e0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.471+0000 7fdcb2000700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 0x7fdc9c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.471+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdca80cd670 con 0x7fdcb41084e0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.472+0000 7fdcb9000700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 0x7fdc9c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.472+0000 7fdcb9000700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 0x7fdc9c0a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4107510 tx=0x7fdca0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:31.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.476+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdca8096360 con 0x7fdcb41084e0 2024-03-11T19:20:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-11T19:20:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-11T19:20:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:31 smithi106 ceph-mon[35889]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:20:31.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-11T19:20:31.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:20:31.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-11T19:20:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:31 smithi143 ceph-mon[36013]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:20:31.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.700+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcb4068930 con 0x7fdcb41084e0 2024-03-11T19:20:31.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.704+0000 7fdcb2000700 1 -- 172.21.15.72:0/1837402500 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24666 (secure 0 0 0) 0x7fdca8080070 con 0x7fdcb41084e0 2024-03-11T19:20:31.709 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:31.709 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":92,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:20:31.311716+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 msgr2=0x7fdc9c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 0x7fdc9c0a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4107510 tx=0x7fdca0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 msgr2=0x7fdcb41a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdca800d980 tx=0x7fdca800dc90 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 shutdown_connections 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdc9c0a1b50 0x7fdc9c0a4010 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb410d5c0 0x7fdcb419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcb41084e0 0x7fdcb41a4420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 --2- 172.21.15.72:0/1837402500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdcb4108e90 0x7fdcb41a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:31.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.707+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 >> 172.21.15.72:0/1837402500 conn(0x7fdcb4075440 msgr2=0x7fdcb4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:31.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.708+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 shutdown_connections 2024-03-11T19:20:31.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:31.708+0000 7fdcbc400700 1 -- 172.21.15.72:0/1837402500 wait complete. 2024-03-11T19:20:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:32 smithi072 ceph-mon[31695]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:20:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:32 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1837402500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:32.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:32 smithi072 ceph-mon[31695]: pgmap v353: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:20:32.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:32 smithi106 ceph-mon[35889]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:20:32.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:32 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1837402500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:32.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:32 smithi106 ceph-mon[35889]: pgmap v353: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:20:32.677 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-11T19:20:32.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:20:32.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:32 smithi143 ceph-mon[36013]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:20:32.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:32 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1837402500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:32.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:32 smithi143 ceph-mon[36013]: pgmap v353: 129 pgs: 129 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:20:33.115 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:35.298 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:35 smithi072 ceph-mon[31695]: pgmap v354: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:35.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:35 smithi106 ceph-mon[35889]: pgmap v354: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:35.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:35 smithi143 ceph-mon[36013]: pgmap v354: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:35.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 -- 172.21.15.72:0/2814785293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c101f20 msgr2=0x7f695c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 --2- 172.21.15.72:0/2814785293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c101f20 0x7f695c102300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69500098f0 tx=0x7f6950009c00 comp rx=0 tx=0).stop 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 -- 172.21.15.72:0/2814785293 shutdown_connections 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 --2- 172.21.15.72:0/2814785293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c106ff0 0x7f695c10ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 --2- 172.21.15.72:0/2814785293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c101f20 0x7f695c102300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 --2- 172.21.15.72:0/2814785293 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c1028d0 0x7f695c106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 -- 172.21.15.72:0/2814785293 >> 172.21.15.72:0/2814785293 conn(0x7f695c078dd0 msgr2=0x7f695c0791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 -- 172.21.15.72:0/2814785293 shutdown_connections 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.599+0000 7f6964800700 1 -- 172.21.15.72:0/2814785293 wait complete. 2024-03-11T19:20:35.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.600+0000 7f6964800700 1 Processor -- start 2024-03-11T19:20:35.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.600+0000 7f6964800700 1 -- start start 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 0x7f695c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19d160 con 0x7f695c106ff0 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19d2d0 con 0x7f695c101f20 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6964800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c19d490 con 0x7f695c1028d0 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6962800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 0x7f695c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6961e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:35.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6961400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6961e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:42012/0 (socket says 172.21.15.72:42012) 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6961400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:59696/0 (socket says 172.21.15.72:59696) 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.601+0000 7f6961e00700 1 -- 172.21.15.72:0/3588558542 learned_addr learned my addr 172.21.15.72:0/3588558542 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 -- 172.21.15.72:0/3588558542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 msgr2=0x7f695c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 -- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 msgr2=0x7f695c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 0x7f695c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 -- 172.21.15.72:0/3588558542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0095f0 con 0x7f695c101f20 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961400700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6962800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 0x7f695c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:35.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6961e00700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6950009230 tx=0x7f6950005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695001c070 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.602+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6950009520 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.603+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f695000ba40 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.603+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950003d20 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.603+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c1a4e00 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.604+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c10a260 con 0x7f695c101f20 2024-03-11T19:20:35.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.606+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f695000b560 con 0x7f695c101f20 2024-03-11T19:20:35.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.607+0000 7f6956a00700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 0x7f69440a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:35.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.607+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f69500ce7c0 con 0x7f695c101f20 2024-03-11T19:20:35.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.608+0000 7f6961400700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 0x7f69440a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:35.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.608+0000 7f6961400700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 0x7f69440a4090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f695c101920 tx=0x7f694c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:35.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.609+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6950097540 con 0x7f695c101f20 2024-03-11T19:20:35.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.827+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f695c1033f0 con 0x7f695c101f20 2024-03-11T19:20:35.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.829+0000 7f6956a00700 1 -- 172.21.15.72:0/3588558542 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24666 (secure 0 0 0) 0x7f6950025090 con 0x7f695c101f20 2024-03-11T19:20:35.834 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:20:35.834 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":92,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:20:31.311716+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:20:35.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 msgr2=0x7f69440a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 0x7f69440a4090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f695c101920 tx=0x7f694c009310 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 msgr2=0x7f695c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6950009230 tx=0x7f6950005040 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 shutdown_connections 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f69440a1bd0 0x7f69440a4090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f695c106ff0 0x7f695c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f695c101f20 0x7f695c198280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 --2- 172.21.15.72:0/3588558542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f695c1028d0 0x7f695c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 >> 172.21.15.72:0/3588558542 conn(0x7f695c078dd0 msgr2=0x7f695c100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 shutdown_connections 2024-03-11T19:20:35.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:35.832+0000 7f6964800700 1 -- 172.21.15.72:0/3588558542 wait complete. 2024-03-11T19:20:36.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:36 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3588558542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:36.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3588558542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:36.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:36 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3588558542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:20:36.771 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-11T19:20:36.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-11T19:20:36.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-11T19:20:36.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-11T19:20:36.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-11T19:20:36.774 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-11T19:20:36.774 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-11T19:20:36.775 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-11T19:20:36.775 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-11T19:20:36.775 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-11T19:20:36.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-11T19:20:36.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-11T19:20:37.434 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:37 smithi072 ceph-mon[31695]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:37.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:37 smithi143 ceph-mon[36013]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:37.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:37 smithi106 ceph-mon[35889]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:37.704 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.406 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:39 smithi072 ceph-mon[31695]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:39 smithi143 ceph-mon[36013]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:39.525 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.525 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.529 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.530 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.537 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.547 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.559 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.565 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.565 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:39.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:39 smithi106 ceph-mon[35889]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-11T19:20:39.686 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:20:41.424 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:41 smithi072 ceph-mon[31695]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 524 B/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-11T19:20:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:41 smithi143 ceph-mon[36013]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 524 B/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-11T19:20:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:41 smithi106 ceph-mon[35889]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 524 B/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 -- 172.21.15.72:0/2545169751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 msgr2=0x7f5f78102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/2545169751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f78102300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009990 tx=0x7f5f68009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 -- 172.21.15.72:0/2545169751 shutdown_connections 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/2545169751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78106ff0 0x7f5f7810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/2545169751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f78102300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/2545169751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 0x7f5f78106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 -- 172.21.15.72:0/2545169751 >> 172.21.15.72:0/2545169751 conn(0x7f5f78078dd0 msgr2=0x7f5f780791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 -- 172.21.15.72:0/2545169751 shutdown_connections 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.586+0000 7f5f7ec00700 1 -- 172.21.15.72:0/2545169751 wait complete. 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 Processor -- start 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 -- start start 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f7819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 0x7f5f7819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78106ff0 0x7f5f7819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819f300 con 0x7f5f78101f20 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819f470 con 0x7f5f78106ff0 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f7ec00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819f630 con 0x7f5f781028d0 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.588+0000 7f5f77e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f7819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f7819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36862/0 (socket says 172.21.15.72:36862) 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 -- 172.21.15.72:0/1552601058 learned_addr learned my addr 172.21.15.72:0/1552601058 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77400700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 0x7f5f7819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:42.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f7cc00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78106ff0 0x7f5f7819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 -- 172.21.15.72:0/1552601058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 msgr2=0x7f5f7819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 0x7f5f7819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 -- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78106ff0 msgr2=0x7f5f7819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78106ff0 0x7f5f7819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680095f0 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77400700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f781028d0 0x7f5f7819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.589+0000 7f5f77e00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f7819a420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005740 tx=0x7f5f68005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.590+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6801d070 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.590+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f6800fb40 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.590+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f680173f0 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.590+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7819f8b0 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.590+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f781a6fa0 con 0x7f5f78101f20 2024-03-11T19:20:42.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.591+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f78111fa0 con 0x7f5f78101f20 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.593+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f5f6800fcb0 con 0x7f5f78101f20 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f74c00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5f5c0a1bd0 0x7f5f5c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5f680cf4e0 con 0x7f5f78101f20 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f77400700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5f5c0a1bd0 0x7f5f5c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f74c00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] conn(0x7f5f5c0acd30 0x7f5f5c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f5c0af8a0 con 0x7f5f5c0acd30 2024-03-11T19:20:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.595+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f680cf8f0 con 0x7f5f78101f20 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.596+0000 7f5f7cc00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] conn(0x7f5f5c0acd30 0x7f5f5c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.596+0000 7f5f77400700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5f5c0a1bd0 0x7f5f5c0a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101b30 tx=0x7f5f64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.596+0000 7f5f7cc00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] conn(0x7f5f5c0acd30 0x7f5f5c0af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.597+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== osd.1 v2:172.21.15.106:6800/1421287783 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5f5c0af8a0 con 0x7f5f5c0acd30 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.615+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 --> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f7810e870 con 0x7f5f5c0acd30 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f74c00700 1 -- 172.21.15.72:0/1552601058 <== osd.1 v2:172.21.15.106:6800/1421287783 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5f7810e870 con 0x7f5f5c0acd30 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] conn(0x7f5f5c0acd30 msgr2=0x7f5f5c0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] conn(0x7f5f5c0acd30 0x7f5f5c0af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5f5c0a1bd0 msgr2=0x7f5f5c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5f5c0a1bd0 0x7f5f5c0a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101b30 tx=0x7f5f64009260 comp rx=0 tx=0).stop 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 msgr2=0x7f5f7819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7ec00700 1 --2- 172.21.15.72:0/1552601058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f78101f20 0x7f5f7819a420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005740 tx=0x7f5f68005770 comp rx=0 tx=0).stop 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.616+0000 7f5f7cc00700 1 -- 172.21.15.72:0/1552601058 reap_dead start 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.617+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 shutdown_connections 2024-03-11T19:20:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.617+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 >> 172.21.15.72:0/1552601058 conn(0x7f5f78078dd0 msgr2=0x7f5f7810de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.617+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 shutdown_connections 2024-03-11T19:20:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:41.617+0000 7f5f7ec00700 1 -- 172.21.15.72:0/1552601058 wait complete. 2024-03-11T19:20:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:43 smithi072 ceph-mon[31695]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 475 B/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-11T19:20:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:43 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:20:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:43 smithi143 ceph-mon[36013]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 475 B/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-11T19:20:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:43 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:20:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:43 smithi106 ceph-mon[35889]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 475 B/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-11T19:20:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:43 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:20:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:45 smithi072 ceph-mon[31695]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-11T19:20:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:45 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:45 smithi143 ceph-mon[36013]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-11T19:20:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:45 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:45 smithi106 ceph-mon[35889]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-11T19:20:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:45 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:20:46.402 INFO:teuthology.orchestra.run.smithi072.stdout:55834574933 2024-03-11T19:20:46.402 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-11T19:20:47.427 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:47 smithi072 ceph-mon[31695]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:47.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:47 smithi143 ceph-mon[36013]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:47.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:47 smithi106 ceph-mon[35889]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:49.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:49 smithi143 ceph-mon[36013]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:49.537 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:49 smithi072 ceph-mon[31695]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:49.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:49 smithi106 ceph-mon[35889]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:51.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:51 smithi106 ceph-mon[35889]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:51.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:51 smithi072 ceph-mon[31695]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:51.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:51 smithi143 ceph-mon[36013]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:53.565 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:53 smithi072 ceph-mon[31695]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:53.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:53 smithi106 ceph-mon[35889]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:53 smithi143 ceph-mon[36013]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 -- 172.21.15.72:0/3745990231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff64073ea0 msgr2=0x7eff64074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 --2- 172.21.15.72:0/3745990231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff64073ea0 0x7eff64074280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff54009990 tx=0x7eff54009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 -- 172.21.15.72:0/3745990231 shutdown_connections 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 --2- 172.21.15.72:0/3745990231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 0x7eff64110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 --2- 172.21.15.72:0/3745990231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff64073ea0 0x7eff64074280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 --2- 172.21.15.72:0/3745990231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff640747c0 0x7eff6410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 -- 172.21.15.72:0/3745990231 >> 172.21.15.72:0/3745990231 conn(0x7eff64078dd0 msgr2=0x7eff640791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 -- 172.21.15.72:0/3745990231 shutdown_connections 2024-03-11T19:20:54.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.699+0000 7eff6b000700 1 -- 172.21.15.72:0/3745990231 wait complete. 2024-03-11T19:20:54.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.700+0000 7eff6b000700 1 Processor -- start 2024-03-11T19:20:54.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.700+0000 7eff6b000700 1 -- start start 2024-03-11T19:20:54.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff64073ea0 0x7eff6419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:54.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff640747c0 0x7eff6419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 0x7eff6419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419f2c0 con 0x7eff640747c0 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419f430 con 0x7eff6410e670 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff6b000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419f5f0 con 0x7eff64073ea0 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff63400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff640747c0 0x7eff6419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff63e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff64073ea0 0x7eff6419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff69000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 0x7eff6419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff63400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff640747c0 0x7eff6419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44836/0 (socket says 172.21.15.72:44836) 2024-03-11T19:20:54.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.701+0000 7eff63400700 1 -- 172.21.15.72:0/2935477176 learned_addr learned my addr 172.21.15.72:0/2935477176 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 -- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff64073ea0 msgr2=0x7eff6419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff64073ea0 0x7eff6419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 -- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff640747c0 msgr2=0x7eff6419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff640747c0 0x7eff6419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540095f0 con 0x7eff6410e670 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff63e00700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff64073ea0 0x7eff6419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff69000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 0x7eff6419ecb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00efc0 tx=0x7eff5c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c018070 con 0x7eff6410e670 2024-03-11T19:20:54.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff5c00f040 con 0x7eff6410e670 2024-03-11T19:20:54.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6419f870 con 0x7eff6410e670 2024-03-11T19:20:54.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd20 con 0x7eff6410e670 2024-03-11T19:20:54.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.702+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a7130 con 0x7eff6410e670 2024-03-11T19:20:54.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.703+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff64114220 con 0x7eff6410e670 2024-03-11T19:20:54.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.706+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7eff5c007500 con 0x7eff6410e670 2024-03-11T19:20:54.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff60c00700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7eff480a1b50 0x7eff480a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:54.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7eff5c0cdd80 con 0x7eff6410e670 2024-03-11T19:20:54.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff63e00700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7eff480a1b50 0x7eff480a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff60c00700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] conn(0x7eff480accb0 0x7eff480af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff480af820 con 0x7eff480accb0 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.707+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c0ce0f0 con 0x7eff6410e670 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.708+0000 7eff63e00700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7eff480a1b50 0x7eff480a4010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff540098e0 tx=0x7eff54005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.708+0000 7eff63400700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] conn(0x7eff480accb0 0x7eff480af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:54.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.708+0000 7eff63400700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] conn(0x7eff480accb0 0x7eff480af150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:54.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.709+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== osd.5 v2:172.21.15.143:6808/1302557280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7eff480af820 con 0x7eff480accb0 2024-03-11T19:20:54.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.727+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 --> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff64062330 con 0x7eff480accb0 2024-03-11T19:20:54.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.728+0000 7eff60c00700 1 -- 172.21.15.72:0/2935477176 <== osd.5 v2:172.21.15.143:6808/1302557280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7eff64062330 con 0x7eff480accb0 2024-03-11T19:20:54.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] conn(0x7eff480accb0 msgr2=0x7eff480af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] conn(0x7eff480accb0 0x7eff480af150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7eff480a1b50 msgr2=0x7eff480a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7eff480a1b50 0x7eff480a4010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff540098e0 tx=0x7eff54005960 comp rx=0 tx=0).stop 2024-03-11T19:20:54.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 msgr2=0x7eff6419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 --2- 172.21.15.72:0/2935477176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff6410e670 0x7eff6419ecb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00efc0 tx=0x7eff5c00c3a0 comp rx=0 tx=0).stop 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff69000700 1 -- 172.21.15.72:0/2935477176 reap_dead start 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 shutdown_connections 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 >> 172.21.15.72:0/2935477176 conn(0x7eff64078dd0 msgr2=0x7eff640fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 shutdown_connections 2024-03-11T19:20:54.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:54.729+0000 7eff6b000700 1 -- 172.21.15.72:0/2935477176 wait complete. 2024-03-11T19:20:55.421 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:55 smithi072 ceph-mon[31695]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:55 smithi106 ceph-mon[35889]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:55 smithi143 ceph-mon[36013]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:20:55.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.806+0000 7fcb3f200700 1 -- 172.21.15.72:0/1969077224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 msgr2=0x7fcb401089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.806+0000 7fcb3f200700 1 --2- 172.21.15.72:0/1969077224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 0x7fcb401089c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009990 tx=0x7fcb2c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 -- 172.21.15.72:0/1969077224 shutdown_connections 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 --2- 172.21.15.72:0/1969077224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb401090a0 0x7fcb40110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 --2- 172.21.15.72:0/1969077224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb401043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 --2- 172.21.15.72:0/1969077224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 0x7fcb401089c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 -- 172.21.15.72:0/1969077224 >> 172.21.15.72:0/1969077224 conn(0x7fcb400ff890 msgr2=0x7fcb40101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 -- 172.21.15.72:0/1969077224 shutdown_connections 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.807+0000 7fcb3f200700 1 -- 172.21.15.72:0/1969077224 wait complete. 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 Processor -- start 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 -- start start 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb4019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 0x7fcb4019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb401090a0 0x7fcb4019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019f120 con 0x7fcb401090a0 2024-03-11T19:20:55.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019f290 con 0x7fcb40103fc0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.808+0000 7fcb3f200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019f450 con 0x7fcb40104970 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb4019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3e800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb401090a0 0x7fcb4019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb4019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40396/0 (socket says 172.21.15.72:40396) 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 -- 172.21.15.72:0/4248118564 learned_addr learned my addr 172.21.15.72:0/4248118564 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3d400700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 0x7fcb4019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 -- 172.21.15.72:0/4248118564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 msgr2=0x7fcb4019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb40104970 0x7fcb4019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 -- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb401090a0 msgr2=0x7fcb4019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb401090a0 0x7fcb4019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c0095f0 con 0x7fcb40103fc0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.809+0000 7fcb3de00700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb4019a240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9a0 tx=0x7fcb3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:55.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.810+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30009790 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.810+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb30010430 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.810+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30018680 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.810+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4019f730 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.810+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb401a6f90 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.811+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb401140a0 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.813+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fcb30010910 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.814+0000 7fcb36a00700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcb240a1b50 0x7fcb240a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.814+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcb30014070 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb3d400700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcb240a1b50 0x7fcb240a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb36a00700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] conn(0x7fcb240accb0 0x7fcb240af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb240af820 con 0x7fcb240accb0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb300ce520 con 0x7fcb40103fc0 2024-03-11T19:20:55.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb3e800700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] conn(0x7fcb240accb0 0x7fcb240af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:55.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.815+0000 7fcb3d400700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcb240a1b50 0x7fcb240a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c006010 tx=0x7fcb2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:55.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.816+0000 7fcb3e800700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] conn(0x7fcb240accb0 0x7fcb240af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:55.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.817+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== osd.6 v2:172.21.15.72:6818/2847471062 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcb240af820 con 0x7fcb240accb0 2024-03-11T19:20:55.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.834+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 --> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb40062380 con 0x7fcb240accb0 2024-03-11T19:20:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.835+0000 7fcb36a00700 1 -- 172.21.15.72:0/4248118564 <== osd.6 v2:172.21.15.72:6818/2847471062 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcb40062380 con 0x7fcb240accb0 2024-03-11T19:20:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.835+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] conn(0x7fcb240accb0 msgr2=0x7fcb240af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.835+0000 7fcb3f200700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] conn(0x7fcb240accb0 0x7fcb240af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcb240a1b50 msgr2=0x7fcb240a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcb240a1b50 0x7fcb240a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c006010 tx=0x7fcb2c005960 comp rx=0 tx=0).stop 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 msgr2=0x7fcb4019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 --2- 172.21.15.72:0/4248118564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb40103fc0 0x7fcb4019a240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9a0 tx=0x7fcb3000ecb0 comp rx=0 tx=0).stop 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3e800700 1 -- 172.21.15.72:0/4248118564 reap_dead start 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 shutdown_connections 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 >> 172.21.15.72:0/4248118564 conn(0x7fcb400ff890 msgr2=0x7fcb40100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:55.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 shutdown_connections 2024-03-11T19:20:55.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:55.836+0000 7fcb3f200700 1 -- 172.21.15.72:0/4248118564 wait complete. 2024-03-11T19:20:57.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.357+0000 7f3f53200700 1 -- 172.21.15.72:0/1936801667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f540761a0 msgr2=0x7f3f54076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 --2- 172.21.15.72:0/1936801667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f540761a0 0x7f3f54076580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009990 tx=0x7f3f44009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 -- 172.21.15.72:0/1936801667 shutdown_connections 2024-03-11T19:20:57.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 --2- 172.21.15.72:0/1936801667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f541129e0 0x7f3f54114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 --2- 172.21.15.72:0/1936801667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f540761a0 0x7f3f54076580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 --2- 172.21.15.72:0/1936801667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f54076ac0 0x7f3f541124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 -- 172.21.15.72:0/1936801667 >> 172.21.15.72:0/1936801667 conn(0x7f3f540ff850 msgr2=0x7f3f54101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 -- 172.21.15.72:0/1936801667 shutdown_connections 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.358+0000 7f3f53200700 1 -- 172.21.15.72:0/1936801667 wait complete. 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.359+0000 7f3f53200700 1 Processor -- start 2024-03-11T19:20:57.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.359+0000 7f3f53200700 1 -- start start 2024-03-11T19:20:57.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 0x7f3f5419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f54076ac0 0x7f3f5419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f541129e0 0x7f3f541a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541a35e0 con 0x7f3f54076ac0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541a3750 con 0x7f3f540761a0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f53200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541a3910 con 0x7f3f541129e0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f4be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f54076ac0 0x7f3f5419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 0x7f3f5419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f52800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f541129e0 0x7f3f541a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f4be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f54076ac0 0x7f3f5419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44888/0 (socket says 172.21.15.72:44888) 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 0x7f3f5419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40420/0 (socket says 172.21.15.72:40420) 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f4be00700 1 -- 172.21.15.72:0/2764525757 learned_addr learned my addr 172.21.15.72:0/2764525757 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 -- 172.21.15.72:0/2764525757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f541129e0 msgr2=0x7f3f541a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f541129e0 0x7f3f541a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 -- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f54076ac0 msgr2=0x7f3f5419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f54076ac0 0x7f3f5419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.360+0000 7f3f51e00700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440095f0 con 0x7f3f540761a0 2024-03-11T19:20:57.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f51e00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 0x7f3f5419e750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f44005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4401d070 con 0x7f3f540761a0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f4400b610 con 0x7f3f540761a0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f541a3b90 con 0x7f3f540761a0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44022e90 con 0x7f3f540761a0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.361+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541ab450 con 0x7f3f540761a0 2024-03-11T19:20:57.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.362+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f54118510 con 0x7f3f540761a0 2024-03-11T19:20:57.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.364+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3f4400fad0 con 0x7f3f540761a0 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4aa00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3f380a1b50 0x7f3f380a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3f440cf3d0 con 0x7f3f540761a0 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4be00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3f380a1b50 0x7f3f380a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4aa00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] conn(0x7f3f380acb40 0x7f3f380aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f380af6b0 con 0x7f3f380acb40 2024-03-11T19:20:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.366+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f440cf7e0 con 0x7f3f540761a0 2024-03-11T19:20:57.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.367+0000 7f3f4be00700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3f380a1b50 0x7f3f380a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f54078fe0 tx=0x7f3f3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:57.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.367+0000 7f3f52800700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] conn(0x7f3f380acb40 0x7f3f380aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:57.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.367+0000 7f3f52800700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] conn(0x7f3f380acb40 0x7f3f380aefe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:57.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.368+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== osd.4 v2:172.21.15.106:6808/1536676428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3f380af6b0 con 0x7f3f380acb40 2024-03-11T19:20:57.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.386+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 --> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f54062380 con 0x7f3f380acb40 2024-03-11T19:20:57.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.387+0000 7f3f4aa00700 1 -- 172.21.15.72:0/2764525757 <== osd.4 v2:172.21.15.106:6808/1536676428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3f54062380 con 0x7f3f380acb40 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.387+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] conn(0x7f3f380acb40 msgr2=0x7f3f380aefe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.387+0000 7f3f53200700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] conn(0x7f3f380acb40 0x7f3f380aefe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3f380a1b50 msgr2=0x7f3f380a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3f380a1b50 0x7f3f380a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f54078fe0 tx=0x7f3f3c009260 comp rx=0 tx=0).stop 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 msgr2=0x7f3f5419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 --2- 172.21.15.72:0/2764525757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f540761a0 0x7f3f5419e750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f44005740 comp rx=0 tx=0).stop 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f52800700 1 -- 172.21.15.72:0/2764525757 reap_dead start 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 shutdown_connections 2024-03-11T19:20:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 >> 172.21.15.72:0/2764525757 conn(0x7f3f540ff850 msgr2=0x7f3f54101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:57.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 shutdown_connections 2024-03-11T19:20:57.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:57.388+0000 7f3f53200700 1 -- 172.21.15.72:0/2764525757 wait complete. 2024-03-11T19:20:57.441 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:57 smithi072 ceph-mon[31695]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:20:57.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:57 smithi106 ceph-mon[35889]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:20:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:57 smithi143 ceph-mon[36013]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:20:59.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:20:59 smithi072 ceph-mon[31695]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.185+0000 7f1162000700 1 -- 172.21.15.72:0/2509179764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c102900 msgr2=0x7f115c1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.185+0000 7f1162000700 1 --2- 172.21.15.72:0/2509179764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c102900 0x7f115c1068a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1148009990 tx=0x7f1148009ca0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 -- 172.21.15.72:0/2509179764 shutdown_connections 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 --2- 172.21.15.72:0/2509179764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c106f80 0x7f115c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 --2- 172.21.15.72:0/2509179764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f115c101f50 0x7f115c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 --2- 172.21.15.72:0/2509179764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c102900 0x7f115c1068a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 -- 172.21.15.72:0/2509179764 >> 172.21.15.72:0/2509179764 conn(0x7f115c078d00 msgr2=0x7f115c079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 -- 172.21.15.72:0/2509179764 shutdown_connections 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.186+0000 7f1162000700 1 -- 172.21.15.72:0/2509179764 wait complete. 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 Processor -- start 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 -- start start 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c101f50 0x7f115c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 0x7f115c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f115c106f80 0x7f115c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a65c0 con 0x7f115c101f50 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a6730 con 0x7f115c102900 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.187+0000 7f1162000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1a68f0 con 0x7f115c106f80 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c101f50 0x7f115c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:59.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 0x7f115c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115be00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f115c106f80 0x7f115c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 0x7f115c0744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40436/0 (socket says 172.21.15.72:40436) 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 -- 172.21.15.72:0/3100972674 learned_addr learned my addr 172.21.15.72:0/3100972674 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 -- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f115c106f80 msgr2=0x7f115c0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f115c106f80 0x7f115c0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 -- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c101f50 msgr2=0x7f115c073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f115c101f50 0x7f115c073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480095f0 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.188+0000 7f115aa00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 0x7f115c0744b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f1148005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.189+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114801d070 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.189+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1a6b70 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.189+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114800b610 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.189+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11480173f0 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.189+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a6fe0 con 0x7f115c102900 2024-03-11T19:20:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.190+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f113c001030 con 0x7f115c102900 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.192+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f114800fad0 con 0x7f115c102900 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.193+0000 7f1153e00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f11400a1bd0 0x7f11400a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.193+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f11480cefb0 con 0x7f115c102900 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.193+0000 7f115b400700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f11400a1bd0 0x7f11400a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.194+0000 7f1153e00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] conn(0x7f11400acd30 0x7f11400af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.194+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11400af8a0 con 0x7f11400acd30 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.194+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11480cf320 con 0x7f115c102900 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.194+0000 7f115b400700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f11400a1bd0 0x7f11400a4090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f114c007be0 tx=0x7f114c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.194+0000 7f115be00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] conn(0x7f11400acd30 0x7f11400af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.195+0000 7f115be00700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] conn(0x7f11400acd30 0x7f11400af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.196+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== osd.2 v2:172.21.15.143:6800/1986223287 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f11400af8a0 con 0x7f11400acd30 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.213+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 --> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f113c002dc0 con 0x7f11400acd30 2024-03-11T19:20:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.214+0000 7f1153e00700 1 -- 172.21.15.72:0/3100972674 <== osd.2 v2:172.21.15.143:6800/1986223287 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f113c002dc0 con 0x7f11400acd30 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] conn(0x7f11400acd30 msgr2=0x7f11400af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] conn(0x7f11400acd30 0x7f11400af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f11400a1bd0 msgr2=0x7f11400a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f11400a1bd0 0x7f11400a4090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f114c007be0 tx=0x7f114c0073d0 comp rx=0 tx=0).stop 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 msgr2=0x7f115c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 --2- 172.21.15.72:0/3100972674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f115c102900 0x7f115c0744b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f1148005740 comp rx=0 tx=0).stop 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f115be00700 1 -- 172.21.15.72:0/3100972674 reap_dead start 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 shutdown_connections 2024-03-11T19:20:59.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 >> 172.21.15.72:0/3100972674 conn(0x7f115c078d00 msgr2=0x7f115c105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:20:59.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 shutdown_connections 2024-03-11T19:20:59.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.215+0000 7f1162000700 1 -- 172.21.15.72:0/3100972674 wait complete. 2024-03-11T19:20:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:20:59 smithi106 ceph-mon[35889]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-11T19:20:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:20:59 smithi143 ceph-mon[36013]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.322+0000 7f7744400700 1 -- 172.21.15.72:0/2771247318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c06adc0 msgr2=0x7f773c1068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.322+0000 7f7744400700 1 --2- 172.21.15.72:0/2771247318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c06adc0 0x7f773c1068c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7728009990 tx=0x7f7728009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 -- 172.21.15.72:0/2771247318 shutdown_connections 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 --2- 172.21.15.72:0/2771247318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c106ff0 0x7f773c10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 --2- 172.21.15.72:0/2771247318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06a410 0x7f773c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 --2- 172.21.15.72:0/2771247318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c06adc0 0x7f773c1068c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 -- 172.21.15.72:0/2771247318 >> 172.21.15.72:0/2771247318 conn(0x7f773c076bc0 msgr2=0x7f773c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 -- 172.21.15.72:0/2771247318 shutdown_connections 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.323+0000 7f7744400700 1 -- 172.21.15.72:0/2771247318 wait complete. 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 Processor -- start 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 -- start start 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 0x7f773c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06adc0 0x7f773c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 0x7f773c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c19f280 con 0x7f773c106ff0 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c19f3f0 con 0x7f773c06adc0 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7744400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c19f5b0 con 0x7f773c06a410 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7742400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 0x7f773c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:01.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7741a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 0x7f773c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7741a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 0x7f773c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:60228/0 (socket says 172.21.15.72:60228) 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7742400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 0x7f773c19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44918/0 (socket says 172.21.15.72:44918) 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7741000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06adc0 0x7f773c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.324+0000 7f7741a00700 1 -- 172.21.15.72:0/2337108743 learned_addr learned my addr 172.21.15.72:0/2337108743 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 -- 172.21.15.72:0/2337108743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 msgr2=0x7f773c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 0x7f773c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 -- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06adc0 msgr2=0x7f773c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06adc0 0x7f773c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280095f0 con 0x7f773c106ff0 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7741a00700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f773c06a410 0x7f773c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7741000700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f773c06adc0 0x7f773c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7742400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 0x7f773c19ec70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f773400d720 tx=0x7f773400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734004220 con 0x7f773c106ff0 2024-03-11T19:21:01.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7734004380 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c19f890 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.325+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734003c30 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.326+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c1a70f0 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.326+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f773c111fd0 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.328+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f773400f690 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.329+0000 7f773a000700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77240a1b50 0x7f77240a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f77340cd650 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f7741a00700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77240a1b50 0x7f77240a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f773a000700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] conn(0x7f77240acc30 0x7f77240af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77240af7a0 con 0x7f77240acc30 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77340cd9c0 con 0x7f773c106ff0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f7741000700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] conn(0x7f77240acc30 0x7f77240af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.330+0000 7f7741a00700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77240a1b50 0x7f77240a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7730009fd0 tx=0x7f7730009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.331+0000 7f7741000700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] conn(0x7f77240acc30 0x7f77240af0d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:01.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.332+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== osd.7 v2:172.21.15.106:6816/4113260481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f77240af7a0 con 0x7f77240acc30 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.349+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 --> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f773c10d8d0 con 0x7f77240acc30 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.349+0000 7f773a000700 1 -- 172.21.15.72:0/2337108743 <== osd.7 v2:172.21.15.106:6816/4113260481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f773c10d8d0 con 0x7f77240acc30 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] conn(0x7f77240acc30 msgr2=0x7f77240af0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] conn(0x7f77240acc30 0x7f77240af0d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77240a1b50 msgr2=0x7f77240a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f77240a1b50 0x7f77240a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7730009fd0 tx=0x7f7730009040 comp rx=0 tx=0).stop 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 msgr2=0x7f773c19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 --2- 172.21.15.72:0/2337108743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f773c106ff0 0x7f773c19ec70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f773400d720 tx=0x7f773400da30 comp rx=0 tx=0).stop 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7742400700 1 -- 172.21.15.72:0/2337108743 reap_dead start 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 shutdown_connections 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 >> 172.21.15.72:0/2337108743 conn(0x7f773c076bc0 msgr2=0x7f773c10cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 shutdown_connections 2024-03-11T19:21:01.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:20:59.350+0000 7f7744400700 1 -- 172.21.15.72:0/2337108743 wait complete. 2024-03-11T19:21:01.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:01 smithi072 ceph-mon[31695]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:01 smithi106 ceph-mon[35889]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:01.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:01 smithi143 ceph-mon[36013]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:02.985 INFO:teuthology.orchestra.run.smithi072.stdout:167503724609 2024-03-11T19:21:02.986 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-11T19:21:03.082 INFO:teuthology.orchestra.run.smithi072.stdout:201863462970 2024-03-11T19:21:03.082 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-11T19:21:03.245 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:03 smithi072 ceph-mon[31695]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:03.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:03 smithi106 ceph-mon[35889]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:03.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:03 smithi143 ceph-mon[36013]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:04.148 INFO:teuthology.orchestra.run.smithi072.stdout:77309411410 2024-03-11T19:21:04.148 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 -- 172.21.15.72:0/2173052770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4102680 msgr2=0x7fe5c410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 --2- 172.21.15.72:0/2173052770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4102680 0x7fe5c410eac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00b0a0 tx=0x7fe5bc00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 -- 172.21.15.72:0/2173052770 shutdown_connections 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 --2- 172.21.15.72:0/2173052770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4102680 0x7fe5c410eac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 --2- 172.21.15.72:0/2173052770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4107ca0 0x7fe5c4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 --2- 172.21.15.72:0/2173052770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5c4101cc0 0x7fe5c4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 -- 172.21.15.72:0/2173052770 >> 172.21.15.72:0/2173052770 conn(0x7fe5c4078da0 msgr2=0x7fe5c40791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 -- 172.21.15.72:0/2173052770 shutdown_connections 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.818+0000 7fe5cb800700 1 -- 172.21.15.72:0/2173052770 wait complete. 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.819+0000 7fe5cb800700 1 Processor -- start 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 -- start start 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5c4101cc0 0x7fe5c4073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 0x7fe5c40744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4107ca0 0x7fe5c40749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41a6640 con 0x7fe5c4101cc0 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41a67b0 con 0x7fe5c4102680 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5cb800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41a6970 con 0x7fe5c4107ca0 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5c8e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5c4101cc0 0x7fe5c4073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5c3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 0x7fe5c40744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5c3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 0x7fe5c40744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:38368/0 (socket says 172.21.15.72:38368) 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.820+0000 7fe5c3e00700 1 -- 172.21.15.72:0/1957321383 learned_addr learned my addr 172.21.15.72:0/1957321383 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c9800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4107ca0 0x7fe5c40749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 -- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4107ca0 msgr2=0x7fe5c40749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5c4107ca0 0x7fe5c40749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 -- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5c4101cc0 msgr2=0x7fe5c4073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5c4101cc0 0x7fe5c4073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009d00 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c3e00700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 0x7fe5c40744b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000e9a0 tx=0x7fe5b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0009790 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0010430 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c41a6c50 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.821+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0018680 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.822+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c41a71a0 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.822+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5c4112000 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.825+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fe5b0010910 con 0x7fe5c4102680 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c1600700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5ac0a1bd0 0x7fe5ac0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe5b0014070 con 0x7fe5c4102680 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c8e00700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5ac0a1bd0 0x7fe5ac0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c1600700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] conn(0x7fe5ac0acd30 0x7fe5ac0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5ac0af8a0 con 0x7fe5ac0acd30 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.826+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5b00ce440 con 0x7fe5c4102680 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.827+0000 7fe5c9800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] conn(0x7fe5ac0acd30 0x7fe5ac0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.827+0000 7fe5c8e00700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5ac0a1bd0 0x7fe5ac0a4090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009be0 tx=0x7fe5b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.827+0000 7fe5c9800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] conn(0x7fe5ac0acd30 0x7fe5ac0af1d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:04.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.828+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== osd.11 v2:172.21.15.143:6824/3564968375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe5ac0af8a0 con 0x7fe5ac0acd30 2024-03-11T19:21:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.846+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 --> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5c4062300 con 0x7fe5ac0acd30 2024-03-11T19:21:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5c1600700 1 -- 172.21.15.72:0/1957321383 <== osd.11 v2:172.21.15.143:6824/3564968375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe5c4062300 con 0x7fe5ac0acd30 2024-03-11T19:21:04.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] conn(0x7fe5ac0acd30 msgr2=0x7fe5ac0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] conn(0x7fe5ac0acd30 0x7fe5ac0af1d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5ac0a1bd0 msgr2=0x7fe5ac0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe5ac0a1bd0 0x7fe5ac0a4090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009be0 tx=0x7fe5b8009260 comp rx=0 tx=0).stop 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 msgr2=0x7fe5c40744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5cb800700 1 --2- 172.21.15.72:0/1957321383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c4102680 0x7fe5c40744b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000e9a0 tx=0x7fe5b000ecb0 comp rx=0 tx=0).stop 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.847+0000 7fe5c9800700 1 -- 172.21.15.72:0/1957321383 reap_dead start 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.848+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 shutdown_connections 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.848+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 >> 172.21.15.72:0/1957321383 conn(0x7fe5c4078da0 msgr2=0x7fe5c4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.848+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 shutdown_connections 2024-03-11T19:21:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:04.848+0000 7fe5cb800700 1 -- 172.21.15.72:0/1957321383 wait complete. 2024-03-11T19:21:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:05 smithi072 ceph-mon[31695]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:05.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:05 smithi106 ceph-mon[35889]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:05 smithi143 ceph-mon[36013]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:07.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:07 smithi072 ceph-mon[31695]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 -- 172.21.15.72:0/899063169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c106ff0 msgr2=0x7f172c10eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 --2- 172.21.15.72:0/899063169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c106ff0 0x7f172c10eb70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f172400b0a0 tx=0x7f172400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 -- 172.21.15.72:0/899063169 shutdown_connections 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 --2- 172.21.15.72:0/899063169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c106ff0 0x7f172c10eb70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 --2- 172.21.15.72:0/899063169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 --2- 172.21.15.72:0/899063169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c06adc0 0x7f172c1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 -- 172.21.15.72:0/899063169 >> 172.21.15.72:0/899063169 conn(0x7f172c076bc0 msgr2=0x7f172c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 -- 172.21.15.72:0/899063169 shutdown_connections 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.236+0000 7f1731800700 1 -- 172.21.15.72:0/899063169 wait complete. 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.237+0000 7f1731800700 1 Processor -- start 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.237+0000 7f1731800700 1 -- start start 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.237+0000 7f1731800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f1731800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c06adc0 0x7f172c1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:07.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f1731800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c106ff0 0x7f172c1016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f1731800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a66d0 con 0x7f172c06adc0 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f1731800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a6840 con 0x7f172c06a410 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f1731800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a6a00 con 0x7f172c106ff0 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c06adc0 0x7f172c1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c06adc0 0x7f172c1011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40878/0 (socket says 172.21.15.72:40878) 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:38382/0 (socket says 172.21.15.72:38382) 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172a000700 1 -- 172.21.15.72:0/4096235089 learned_addr learned my addr 172.21.15.72:0/4096235089 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 -- 172.21.15.72:0/4096235089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c106ff0 msgr2=0x7f172c1016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172b400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c106ff0 0x7f172c1016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c106ff0 0x7f172c1016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 -- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c06adc0 msgr2=0x7f172c1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f172c06adc0 0x7f172c1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172aa00700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724009d00 con 0x7f172c06a410 2024-03-11T19:21:07.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.238+0000 7f172b400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172c106ff0 0x7f172c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f172aa00700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c100c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9f0 tx=0x7f171c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0096d0 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f171c00cbd0 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c1a6ce0 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c018680 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.239+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c1a7190 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.240+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f172c111fc0 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.242+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f171c00cd40 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.243+0000 7f1723400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17100a1b50 0x7f17100a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f171c014070 con 0x7f172c06a410 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f172a000700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17100a1b50 0x7f17100a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:07.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f1723400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] conn(0x7f17100accb0 0x7f17100af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17100af820 con 0x7f17100accb0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171c0d22a0 con 0x7f172c06a410 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.244+0000 7f172b400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] conn(0x7f17100accb0 0x7f17100af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.245+0000 7f172a000700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17100a1b50 0x7f17100a4010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f172c100730 tx=0x7f1718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.245+0000 7f172b400700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] conn(0x7f17100accb0 0x7f17100af150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.246+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== osd.0 v2:172.21.15.72:6802/1634362166 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f17100af820 con 0x7f17100accb0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.273+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 --> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f172c062380 con 0x7f17100accb0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.274+0000 7f1723400700 1 -- 172.21.15.72:0/4096235089 <== osd.0 v2:172.21.15.72:6802/1634362166 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f172c062380 con 0x7f17100accb0 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] conn(0x7f17100accb0 msgr2=0x7f17100af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] conn(0x7f17100accb0 0x7f17100af150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17100a1b50 msgr2=0x7f17100a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f17100a1b50 0x7f17100a4010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f172c100730 tx=0x7f1718009260 comp rx=0 tx=0).stop 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 msgr2=0x7f172c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 --2- 172.21.15.72:0/4096235089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f172c06a410 0x7f172c100c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9f0 tx=0x7f171c00ed00 comp rx=0 tx=0).stop 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f172b400700 1 -- 172.21.15.72:0/4096235089 reap_dead start 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 shutdown_connections 2024-03-11T19:21:07.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 >> 172.21.15.72:0/4096235089 conn(0x7f172c076bc0 msgr2=0x7f172c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:07.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 shutdown_connections 2024-03-11T19:21:07.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.275+0000 7f1731800700 1 -- 172.21.15.72:0/4096235089 wait complete. 2024-03-11T19:21:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:07 smithi106 ceph-mon[35889]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:07.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:07 smithi143 ceph-mon[36013]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:08.303 INFO:teuthology.orchestra.run.smithi072.stdout:347892351008 2024-03-11T19:21:08.303 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-11T19:21:08.580 INFO:teuthology.orchestra.run.smithi072.stdout:223338299445 2024-03-11T19:21:08.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1546035057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae01026e0 msgr2=0x7f4ae010ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1546035057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae01026e0 0x7f4ae010ead0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009990 tx=0x7f4ad8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1546035057 shutdown_connections 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1546035057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae01026e0 0x7f4ae010ead0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1546035057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae0107d90 0x7f4ae0108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1546035057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ae0101d20 0x7f4ae01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.384+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1546035057 >> 172.21.15.72:0/1546035057 conn(0x7f4ae0078d40 msgr2=0x7f4ae0079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.385+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1546035057 shutdown_connections 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.385+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1546035057 wait complete. 2024-03-11T19:21:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 Processor -- start 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 -- start start 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 0x7f4ae0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ae01026e0 0x7f4ae0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae0107d90 0x7f4ae019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae019d010 con 0x7f4ae0101d20 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae019d180 con 0x7f4ae01026e0 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae7e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae019d340 con 0x7f4ae0107d90 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae5400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 0x7f4ae0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.386+0000 7f4ae5400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 0x7f4ae0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40884/0 (socket says 172.21.15.72:40884) 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 -- 172.21.15.72:0/1269753295 learned_addr learned my addr 172.21.15.72:0/1269753295 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae4a00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ae01026e0 0x7f4ae0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:09.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae0107d90 0x7f4ae019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 -- 172.21.15.72:0/1269753295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae0107d90 msgr2=0x7f4ae019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae0107d90 0x7f4ae019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 -- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ae01026e0 msgr2=0x7f4ae0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ae01026e0 0x7f4ae0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80095f0 con 0x7f4ae0101d20 2024-03-11T19:21:09.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.387+0000 7f4ae5400700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 0x7f4ae0198130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400ef10 tx=0x7f4ad400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.388+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400c9c0 con 0x7f4ae0101d20 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.388+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ad400f040 con 0x7f4ae0101d20 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.388+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae019d5c0 con 0x7f4ae0101d20 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.388+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4014480 con 0x7f4ae0101d20 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.388+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae01a4e80 con 0x7f4ae0101d20 2024-03-11T19:21:09.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.389+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ae0112050 con 0x7f4ae0101d20 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.391+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4ad4007720 con 0x7f4ae0101d20 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ade000700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ac80a1bd0 0x7f4ac80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4ad40cdb50 con 0x7f4ae0101d20 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ae4a00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ac80a1bd0 0x7f4ac80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ade000700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] conn(0x7f4ac80acd30 0x7f4ac80af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:09.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ac80af8a0 con 0x7f4ac80acd30 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.392+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ad40d22a0 con 0x7f4ae0101d20 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.393+0000 7f4ae5e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] conn(0x7f4ac80acd30 0x7f4ac80af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.393+0000 7f4ae4a00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ac80a1bd0 0x7f4ac80a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00792d0 tx=0x7f4acc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.393+0000 7f4ae5e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] conn(0x7f4ac80acd30 0x7f4ac80af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.394+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== osd.10 v2:172.21.15.106:6824/3578514696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4ac80af8a0 con 0x7f4ac80acd30 2024-03-11T19:21:09.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.412+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 --> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ae0062330 con 0x7f4ac80acd30 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.413+0000 7f4ade000700 1 -- 172.21.15.72:0/1269753295 <== osd.10 v2:172.21.15.106:6824/3578514696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4ae0062330 con 0x7f4ac80acd30 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.413+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] conn(0x7f4ac80acd30 msgr2=0x7f4ac80af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.413+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] conn(0x7f4ac80acd30 0x7f4ac80af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ac80a1bd0 msgr2=0x7f4ac80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4ac80a1bd0 0x7f4ac80a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00792d0 tx=0x7f4acc009260 comp rx=0 tx=0).stop 2024-03-11T19:21:09.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 msgr2=0x7f4ae0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 --2- 172.21.15.72:0/1269753295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ae0101d20 0x7f4ae0198130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400ef10 tx=0x7f4ad400c3a0 comp rx=0 tx=0).stop 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae5e00700 1 -- 172.21.15.72:0/1269753295 reap_dead start 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 shutdown_connections 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 >> 172.21.15.72:0/1269753295 conn(0x7f4ae0078d40 msgr2=0x7f4ae01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 shutdown_connections 2024-03-11T19:21:09.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:07.414+0000 7f4ae7e00700 1 -- 172.21.15.72:0/1269753295 wait complete. 2024-03-11T19:21:09.454 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:09 smithi072 ceph-mon[31695]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:21:09.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:09 smithi106 ceph-mon[35889]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:21:09.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:09 smithi143 ceph-mon[36013]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:21:10.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 -- 172.21.15.72:0/445903428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9948108d10 msgr2=0x7f99481090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 --2- 172.21.15.72:0/445903428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9948108d10 0x7f99481090f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9938009990 tx=0x7f9938009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 -- 172.21.15.72:0/445903428 shutdown_connections 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 --2- 172.21.15.72:0/445903428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f994810d6c0 0x7f9948115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 --2- 172.21.15.72:0/445903428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9948108d10 0x7f99481090f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 --2- 172.21.15.72:0/445903428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99481096c0 0x7f994810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 -- 172.21.15.72:0/445903428 >> 172.21.15.72:0/445903428 conn(0x7f9948077840 msgr2=0x7f9948079c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 -- 172.21.15.72:0/445903428 shutdown_connections 2024-03-11T19:21:10.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.292+0000 7f994d600700 1 -- 172.21.15.72:0/445903428 wait complete. 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.293+0000 7f994d600700 1 Processor -- start 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.293+0000 7f994d600700 1 -- start start 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9948108d10 0x7f99481a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99481096c0 0x7f99481a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 0x7f994819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819eef0 con 0x7f994810d6c0 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819f060 con 0x7f9948108d10 2024-03-11T19:21:10.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994d600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819f220 con 0x7f99481096c0 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 0x7f994819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994cc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 0x7f994819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40920/0 (socket says 172.21.15.72:40920) 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f994cc00700 1 -- 172.21.15.72:0/2773631219 learned_addr learned my addr 172.21.15.72:0/2773631219 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f9947e00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9948108d10 0x7f99481a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.294+0000 7f9947400700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99481096c0 0x7f99481a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 -- 172.21.15.72:0/2773631219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99481096c0 msgr2=0x7f99481a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99481096c0 0x7f99481a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 -- 172.21.15.72:0/2773631219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9948108d10 msgr2=0x7f99481a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9948108d10 0x7f99481a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380095f0 con 0x7f994810d6c0 2024-03-11T19:21:10.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f9947e00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9948108d10 0x7f99481a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994cc00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 0x7f994819e740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9940007f10 tx=0x7f994000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940017070 con 0x7f994810d6c0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f994000f040 con 0x7f994810d6c0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994819f500 con 0x7f994810d6c0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99481ab5f0 con 0x7f994810d6c0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.295+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940005760 con 0x7f994810d6c0 2024-03-11T19:21:10.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.296+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9948109b50 con 0x7f994810d6c0 2024-03-11T19:21:10.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.299+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f99400058c0 con 0x7f994810d6c0 2024-03-11T19:21:10.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.300+0000 7f9944c00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f992c0a1bd0 0x7f992c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:10.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.300+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f99400cd670 con 0x7f994810d6c0 2024-03-11T19:21:10.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.300+0000 7f9947e00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f992c0a1bd0 0x7f992c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9944c00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] conn(0x7f992c0acd30 0x7f992c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f992c0af8a0 con 0x7f992c0acd30 2024-03-11T19:21:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99400cd9e0 con 0x7f994810d6c0 2024-03-11T19:21:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9947400700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] conn(0x7f992c0acd30 0x7f992c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9947e00700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f992c0a1bd0 0x7f992c0a4090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9938006010 tx=0x7f993800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:10.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.301+0000 7f9947400700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] conn(0x7f992c0acd30 0x7f992c0af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:10.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.303+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== osd.3 v2:172.21.15.72:6810/3085969755 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f992c0af8a0 con 0x7f992c0acd30 2024-03-11T19:21:10.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.320+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 --> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9948118770 con 0x7f992c0acd30 2024-03-11T19:21:10.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.321+0000 7f9944c00700 1 -- 172.21.15.72:0/2773631219 <== osd.3 v2:172.21.15.72:6810/3085969755 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9948118770 con 0x7f992c0acd30 2024-03-11T19:21:10.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.321+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] conn(0x7f992c0acd30 msgr2=0x7f992c0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.321+0000 7f994d600700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] conn(0x7f992c0acd30 0x7f992c0af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f992c0a1bd0 msgr2=0x7f992c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f992c0a1bd0 0x7f992c0a4090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9938006010 tx=0x7f993800b540 comp rx=0 tx=0).stop 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 msgr2=0x7f994819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 --2- 172.21.15.72:0/2773631219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f994810d6c0 0x7f994819e740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9940007f10 tx=0x7f994000d3b0 comp rx=0 tx=0).stop 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994cc00700 1 -- 172.21.15.72:0/2773631219 reap_dead start 2024-03-11T19:21:10.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 shutdown_connections 2024-03-11T19:21:10.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 >> 172.21.15.72:0/2773631219 conn(0x7f9948077840 msgr2=0x7f9948079c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:10.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 shutdown_connections 2024-03-11T19:21:10.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:10.322+0000 7f994d600700 1 -- 172.21.15.72:0/2773631219 wait complete. 2024-03-11T19:21:10.652 INFO:teuthology.orchestra.run.smithi072.stdout:146028888135 2024-03-11T19:21:10.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-11T19:21:11.052 INFO:teuthology.orchestra.run.smithi072.stdout:34359738463 2024-03-11T19:21:11.053 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-11T19:21:11.253 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:11 smithi072 ceph-mon[31695]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:11.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 -- 172.21.15.72:0/1348390006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc81084e0 msgr2=0x7f8cc81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 --2- 172.21.15.72:0/1348390006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc81084e0 0x7f8cc81088c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009990 tx=0x7f8cb8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 -- 172.21.15.72:0/1348390006 shutdown_connections 2024-03-11T19:21:11.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 --2- 172.21.15.72:0/1348390006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cc810d5c0 0x7f8cc8115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 --2- 172.21.15.72:0/1348390006 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc81084e0 0x7f8cc81088c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 --2- 172.21.15.72:0/1348390006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cc8108e90 0x7f8cc810cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.619+0000 7f8cce600700 1 -- 172.21.15.72:0/1348390006 >> 172.21.15.72:0/1348390006 conn(0x7f8cc8075440 msgr2=0x7f8cc8077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.620+0000 7f8cce600700 1 -- 172.21.15.72:0/1348390006 shutdown_connections 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.620+0000 7f8cce600700 1 -- 172.21.15.72:0/1348390006 wait complete. 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 Processor -- start 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 -- start start 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cc81084e0 0x7f8cc81a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cc8108e90 0x7f8cc81a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 0x7f8cc819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc819ec90 con 0x7f8cc810d5c0 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc819edd0 con 0x7f8cc8108e90 2024-03-11T19:21:11.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.621+0000 7f8cce600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc819ef90 con 0x7f8cc81084e0 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 0x7f8cc819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 0x7f8cc819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40900/0 (socket says 172.21.15.72:40900) 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 -- 172.21.15.72:0/1971456657 learned_addr learned my addr 172.21.15.72:0/1971456657 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7400700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cc81084e0 0x7f8cc81a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc6a00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cc8108e90 0x7f8cc81a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 -- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cc81084e0 msgr2=0x7f8cc81a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cc81084e0 0x7f8cc81a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 -- 172.21.15.72:0/1971456657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cc8108e90 msgr2=0x7f8cc81a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cc8108e90 0x7f8cc81a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80095f0 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.622+0000 7f8cc7e00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 0x7f8cc819e630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000ef30 tx=0x7f8cc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.623+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0018070 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.623+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc000f040 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.623+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc819f210 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.623+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000cd50 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.623+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc81ab4c0 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.624+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cc81099c0 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.626+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f8cc0007500 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cbfe00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8cac0a1bd0 0x7f8cac0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8cc00cdda0 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cc7400700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8cac0a1bd0 0x7f8cac0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cbfe00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] conn(0x7f8cac0acd30 0x7f8cac0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cac0af8a0 con 0x7f8cac0acd30 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.627+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc00ce110 con 0x7f8cc810d5c0 2024-03-11T19:21:11.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.628+0000 7f8cc6a00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] conn(0x7f8cac0acd30 0x7f8cac0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.628+0000 7f8cc7400700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8cac0a1bd0 0x7f8cac0a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098e0 tx=0x7f8cb8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.628+0000 7f8cc6a00700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] conn(0x7f8cac0acd30 0x7f8cac0af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.629+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== osd.8 v2:172.21.15.143:6816/1425068959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8cac0af8a0 con 0x7f8cac0acd30 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.647+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 --> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cc8118620 con 0x7f8cac0acd30 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.648+0000 7f8cbfe00700 1 -- 172.21.15.72:0/1971456657 <== osd.8 v2:172.21.15.143:6816/1425068959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8cc8118620 con 0x7f8cac0acd30 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] conn(0x7f8cac0acd30 msgr2=0x7f8cac0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] conn(0x7f8cac0acd30 0x7f8cac0af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8cac0a1bd0 msgr2=0x7f8cac0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f8cac0a1bd0 0x7f8cac0a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098e0 tx=0x7f8cb8005960 comp rx=0 tx=0).stop 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 msgr2=0x7f8cc819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 --2- 172.21.15.72:0/1971456657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8cc810d5c0 0x7f8cc819e630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000ef30 tx=0x7f8cc000c3a0 comp rx=0 tx=0).stop 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cc7e00700 1 -- 172.21.15.72:0/1971456657 reap_dead start 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 shutdown_connections 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 >> 172.21.15.72:0/1971456657 conn(0x7f8cc8075440 msgr2=0x7f8cc8077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 shutdown_connections 2024-03-11T19:21:11.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:09.649+0000 7f8cce600700 1 -- 172.21.15.72:0/1971456657 wait complete. 2024-03-11T19:21:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:11 smithi106 ceph-mon[35889]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:11 smithi143 ceph-mon[36013]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:12.731 INFO:teuthology.orchestra.run.smithi072.stdout:257698037810 2024-03-11T19:21:12.732 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-11T19:21:13.492 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:13 smithi143 ceph-mon[36013]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:21:13.632 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:13 smithi072 ceph-mon[31695]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:21:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:13 smithi106 ceph-mon[35889]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:21:13.684 INFO:teuthology.orchestra.run.smithi072.stdout:313532612644 2024-03-11T19:21:13.684 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-11T19:21:14.735 INFO:teuthology.orchestra.run.smithi072.stdout:111669149774 2024-03-11T19:21:14.736 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 -- 172.21.15.72:0/254521638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c076ac0 msgr2=0x7fdd8c1124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 --2- 172.21.15.72:0/254521638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c076ac0 0x7fdd8c1124c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009990 tx=0x7fdd74009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 -- 172.21.15.72:0/254521638 shutdown_connections 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 --2- 172.21.15.72:0/254521638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c112a00 0x7fdd8c114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 --2- 172.21.15.72:0/254521638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 --2- 172.21.15.72:0/254521638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c076ac0 0x7fdd8c1124c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 -- 172.21.15.72:0/254521638 >> 172.21.15.72:0/254521638 conn(0x7fdd8c0ff890 msgr2=0x7fdd8c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.284+0000 7fdd90a00700 1 -- 172.21.15.72:0/254521638 shutdown_connections 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.285+0000 7fdd90a00700 1 -- 172.21.15.72:0/254521638 wait complete. 2024-03-11T19:21:14.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 Processor -- start 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 -- start start 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c076ac0 0x7fdd8c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c112a00 0x7fdd8c1a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1a35b0 con 0x7fdd8c0761a0 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1a3720 con 0x7fdd8c112a00 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd90a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1a38e0 con 0x7fdd8c076ac0 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd8a800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd8a800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56822/0 (socket says 172.21.15.72:56822) 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.286+0000 7fdd8a800700 1 -- 172.21.15.72:0/3812614484 learned_addr learned my addr 172.21.15.72:0/3812614484 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd81e00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c076ac0 0x7fdd8c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8b200700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c112a00 0x7fdd8c1a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 -- 172.21.15.72:0/3812614484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c076ac0 msgr2=0x7fdd8c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c076ac0 0x7fdd8c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 -- 172.21.15.72:0/3812614484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c112a00 msgr2=0x7fdd8c1a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c112a00 0x7fdd8c1a2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740095f0 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd8a800700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c19e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e9a0 tx=0x7fdd7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009790 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.287+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd7c010430 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.288+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c1a3bc0 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.288+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c018680 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.288+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c1ab420 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.289+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd8c1185b0 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.291+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fdd7c010910 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.292+0000 7fdd83e00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd700a1bd0 0x7fdd700a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.292+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdd7c014070 con 0x7fdd8c0761a0 2024-03-11T19:21:14.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.292+0000 7fdd83e00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] conn(0x7fdd700acd30 0x7fdd700af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.292+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd700af8a0 con 0x7fdd700acd30 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.292+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7c0ce470 con 0x7fdd8c0761a0 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.293+0000 7fdd81e00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd700a1bd0 0x7fdd700a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.293+0000 7fdd8b200700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] conn(0x7fdd700acd30 0x7fdd700af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.293+0000 7fdd81e00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd700a1bd0 0x7fdd700a4090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1032f0 tx=0x7fdd7400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:14.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.293+0000 7fdd8b200700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] conn(0x7fdd700acd30 0x7fdd700af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.294+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== osd.9 v2:172.21.15.72:6826/98146808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdd700af8a0 con 0x7fdd700acd30 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.312+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 --> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd8c114db0 con 0x7fdd700acd30 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.313+0000 7fdd83e00700 1 -- 172.21.15.72:0/3812614484 <== osd.9 v2:172.21.15.72:6826/98146808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdd8c114db0 con 0x7fdd700acd30 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] conn(0x7fdd700acd30 msgr2=0x7fdd700af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] conn(0x7fdd700acd30 0x7fdd700af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd700a1bd0 msgr2=0x7fdd700a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd700a1bd0 0x7fdd700a4090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1032f0 tx=0x7fdd7400b260 comp rx=0 tx=0).stop 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 msgr2=0x7fdd8c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 --2- 172.21.15.72:0/3812614484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd8c0761a0 0x7fdd8c19e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e9a0 tx=0x7fdd7c00ecb0 comp rx=0 tx=0).stop 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd8b200700 1 -- 172.21.15.72:0/3812614484 reap_dead start 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 shutdown_connections 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 >> 172.21.15.72:0/3812614484 conn(0x7fdd8c0ff890 msgr2=0x7fdd8c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 shutdown_connections 2024-03-11T19:21:14.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:14.314+0000 7fdd90a00700 1 -- 172.21.15.72:0/3812614484 wait complete. 2024-03-11T19:21:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:15 smithi106 ceph-mon[35889]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:15.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:15 smithi072 ceph-mon[31695]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:15.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:15 smithi143 ceph-mon[36013]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:15.849 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.859 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.864 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.868 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.876 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.877 INFO:teuthology.orchestra.run.smithi072.stdout:292057776171 2024-03-11T19:21:15.877 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-11T19:21:15.940 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:15.957 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:16.365 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:17.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:17 smithi072 ceph-mon[31695]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:17 smithi106 ceph-mon[35889]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:17 smithi143 ceph-mon[36013]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:19.493 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:19.502 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:19.504 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:19.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:19 smithi072 ceph-mon[31695]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:19 smithi106 ceph-mon[35889]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:19.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:19 smithi143 ceph-mon[36013]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:21.513 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:21 smithi072 ceph-mon[31695]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-11T19:21:21.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:21 smithi106 ceph-mon[35889]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-11T19:21:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:21 smithi143 ceph-mon[36013]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-03-11T19:21:23.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:23 smithi072 ceph-mon[31695]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:23 smithi106 ceph-mon[35889]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:23 smithi143 ceph-mon[36013]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:21:25.531 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:25 smithi072 ceph-mon[31695]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:21:25.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:25 smithi106 ceph-mon[35889]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:21:25.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:25 smithi143 ceph-mon[36013]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:21:27.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:27 smithi072 ceph-mon[31695]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:27.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:27 smithi106 ceph-mon[35889]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:27.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:27 smithi143 ceph-mon[36013]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:29.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:29 smithi072 ceph-mon[31695]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:29.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:29 smithi106 ceph-mon[35889]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:29.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:29 smithi143 ceph-mon[36013]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:30.411 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:30 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2608355041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:21:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:30 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2608355041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:21:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:30 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2608355041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:21:31.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:31 smithi072 ceph-mon[31695]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:31.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:31 smithi072 ceph-mon[31695]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:21:31.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:31 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-11T19:21:31.520 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:31 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3087349747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 -- 172.21.15.72:0/3230304368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e00747c0 msgr2=0x7f04e010e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 --2- 172.21.15.72:0/3230304368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e00747c0 0x7f04e010e130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009990 tx=0x7f04cc009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 -- 172.21.15.72:0/3230304368 shutdown_connections 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 --2- 172.21.15.72:0/3230304368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e010e670 0x7f04e0110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 --2- 172.21.15.72:0/3230304368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e0073ea0 0x7f04e0074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 --2- 172.21.15.72:0/3230304368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e00747c0 0x7f04e010e130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 -- 172.21.15.72:0/3230304368 >> 172.21.15.72:0/3230304368 conn(0x7f04e0078dd0 msgr2=0x7f04e00791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 -- 172.21.15.72:0/3230304368 shutdown_connections 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.055+0000 7f04e5800700 1 -- 172.21.15.72:0/3230304368 wait complete. 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.056+0000 7f04e5800700 1 Processor -- start 2024-03-11T19:21:31.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 -- start start 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e00747c0 0x7f04e019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e010e670 0x7f04e019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e019f380 con 0x7f04e010e670 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e019f4f0 con 0x7f04e00747c0 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04e5800700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e019f6b0 con 0x7f04e0073ea0 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04dea00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04dea00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:41276/0 (socket says 172.21.15.72:41276) 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04dea00700 1 -- 172.21.15.72:0/2608355041 learned_addr learned my addr 172.21.15.72:0/2608355041 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04df400700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e010e670 0x7f04e019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.057+0000 7f04de000700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e00747c0 0x7f04e019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 -- 172.21.15.72:0/2608355041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e00747c0 msgr2=0x7f04e019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e00747c0 0x7f04e019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 -- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e010e670 msgr2=0x7f04e019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e010e670 0x7f04e019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 -- 172.21.15.72:0/2608355041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0095f0 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04dea00700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04d000d950 tx=0x7f04d000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000f890 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04d0010460 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e019f990 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000e610 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.058+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e01a71f0 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.059+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e0109880 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.061+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f04d0010960 con 0x7f04e0073ea0 2024-03-11T19:21:31.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.063+0000 7f04d7400700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 0x7f04c40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stdout:146028888141 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.063+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f04d00cd6d0 con 0x7f04e0073ea0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.063+0000 7f04de000700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 0x7f04c40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.064+0000 7f04de000700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 0x7f04c40a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04e0110370 tx=0x7f04cc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.065+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04d00963c0 con 0x7f04e0073ea0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.297+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f04e019b800 con 0x7f04e0073ea0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.298+0000 7f04d7400700 1 -- 172.21.15.72:0/2608355041 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04d0020020 con 0x7f04e0073ea0 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 msgr2=0x7f04c40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 0x7f04c40a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04e0110370 tx=0x7f04cc00b260 comp rx=0 tx=0).stop 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 msgr2=0x7f04e019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:31.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04d000d950 tx=0x7f04d000dc60 comp rx=0 tx=0).stop 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 shutdown_connections 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04c40a1b50 0x7f04c40a4010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04e010e670 0x7f04e019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04e0073ea0 0x7f04e019a4a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 --2- 172.21.15.72:0/2608355041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04e00747c0 0x7f04e019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 >> 172.21.15.72:0/2608355041 conn(0x7f04e0078dd0 msgr2=0x7f04e00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.301+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 shutdown_connections 2024-03-11T19:21:31.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:29.302+0000 7f04e5800700 1 -- 172.21.15.72:0/2608355041 wait complete. 2024-03-11T19:21:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:31 smithi106 ceph-mon[35889]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:31 smithi106 ceph-mon[35889]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:21:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:31 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-11T19:21:31.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:31 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3087349747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:21:31.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:31 smithi143 ceph-mon[36013]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:31 smithi143 ceph-mon[36013]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:21:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:31 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-11T19:21:31.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:31 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3087349747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 -- 172.21.15.72:0/2660656114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 msgr2=0x7fa5dc108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/2660656114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc108170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009990 tx=0x7fa5d0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 -- 172.21.15.72:0/2660656114 shutdown_connections 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/2660656114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/2660656114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc108170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/2660656114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc101d20 0x7fa5dc1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 -- 172.21.15.72:0/2660656114 >> 172.21.15.72:0/2660656114 conn(0x7fa5dc078d40 msgr2=0x7fa5dc079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.187+0000 7fa5e3c00700 1 -- 172.21.15.72:0/2660656114 shutdown_connections 2024-03-11T19:21:32.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.188+0000 7fa5e3c00700 1 -- 172.21.15.72:0/2660656114 wait complete. 2024-03-11T19:21:32.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 Processor -- start 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 -- start start 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 0x7fa5dc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc19f2b0 con 0x7fa5dc107d90 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc19f420 con 0x7fa5dc101d20 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e3c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc19f5e0 con 0x7fa5dc1026e0 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.189+0000 7fa5e1c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48366/0 (socket says 172.21.15.72:48366) 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 -- 172.21.15.72:0/3208433136 learned_addr learned my addr 172.21.15.72:0/3208433136 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5dbe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1200700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 0x7fa5dc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 -- 172.21.15.72:0/3208433136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 msgr2=0x7fa5dc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 -- 172.21.15.72:0/3208433136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 msgr2=0x7fa5dc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 0x7fa5dc19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 -- 172.21.15.72:0/3208433136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d00095f0 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5dbe00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1200700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 0x7fa5dc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.190+0000 7fa5e1c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400ef30 tx=0x7fa5d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.191+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4018070 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.191+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5d400f040 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.191+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc19f860 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.191+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400cd70 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.191+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc1a7120 con 0x7fa5dc107d90 2024-03-11T19:21:32.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.192+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc068930 con 0x7fa5dc107d90 2024-03-11T19:21:32.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.194+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa5d4007500 con 0x7fa5dc107d90 2024-03-11T19:21:32.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.195+0000 7fa5d9600700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 0x7fa5c40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.195+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa5d40cd310 con 0x7fa5dc107d90 2024-03-11T19:21:32.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.195+0000 7fa5e1200700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 0x7fa5c40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.196+0000 7fa5e1200700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 0x7fa5c40a3f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000b2e0 tx=0x7fa5d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:32.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.200+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5d40961d0 con 0x7fa5dc107d90 2024-03-11T19:21:32.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.434+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5dc111da0 con 0x7fa5dc107d90 2024-03-11T19:21:32.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.435+0000 7fa5d9600700 1 -- 172.21.15.72:0/3208433136 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5d4099890 con 0x7fa5dc107d90 2024-03-11T19:21:32.439 INFO:teuthology.orchestra.run.smithi072.stdout:77309411416 2024-03-11T19:21:32.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 msgr2=0x7fa5c40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 0x7fa5c40a3f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000b2e0 tx=0x7fa5d0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 msgr2=0x7fa5dc19eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400ef30 tx=0x7fa5d400c3a0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 shutdown_connections 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa5c40a1ac0 0x7fa5c40a3f80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc107d90 0x7fa5dc19eca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5dc101d20 0x7fa5dc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 --2- 172.21.15.72:0/3208433136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5dc1026e0 0x7fa5dc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 >> 172.21.15.72:0/3208433136 conn(0x7fa5dc078d40 msgr2=0x7fa5dc100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 shutdown_connections 2024-03-11T19:21:32.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:32.438+0000 7fa5e3c00700 1 -- 172.21.15.72:0/3208433136 wait complete. 2024-03-11T19:21:32.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 -- 172.21.15.72:0/1379521056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 msgr2=0x7f34541044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 --2- 172.21.15.72:0/1379521056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f34541044a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 -- 172.21.15.72:0/1379521056 shutdown_connections 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 --2- 172.21.15.72:0/1379521056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34541091a0 0x7f3454110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 --2- 172.21.15.72:0/1379521056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f34541044a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 --2- 172.21.15.72:0/1379521056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3454104a70 0x7f3454108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 -- 172.21.15.72:0/1379521056 >> 172.21.15.72:0/1379521056 conn(0x7f34540ff950 msgr2=0x7f3454101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 -- 172.21.15.72:0/1379521056 shutdown_connections 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.904+0000 7f345a400700 1 -- 172.21.15.72:0/1379521056 wait complete. 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.905+0000 7f345a400700 1 Processor -- start 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.905+0000 7f345a400700 1 -- start start 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 0x7f345419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34541091a0 0x7f345419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345419f2f0 con 0x7f34541040c0 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345419f460 con 0x7f3454104a70 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f345a400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345419f620 con 0x7f34541091a0 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f3453400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f3453400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60564/0 (socket says 172.21.15.72:60564) 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f3453400700 1 -- 172.21.15.72:0/3087349747 learned_addr learned my addr 172.21.15.72:0/3087349747 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f344be00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 0x7f345419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.906+0000 7f3453e00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34541091a0 0x7f345419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 -- 172.21.15.72:0/3087349747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34541091a0 msgr2=0x7f345419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34541091a0 0x7f345419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 -- 172.21.15.72:0/3087349747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 msgr2=0x7f345419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 0x7f345419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 -- 172.21.15.72:0/3087349747 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440095f0 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f344be00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 0x7f345419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3453400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f3444005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344401d070 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345419f8a0 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f344400b610 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440173f0 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.907+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541a7160 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.908+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345410c460 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.910+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3444022690 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.911+0000 7f3450c00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 0x7f34380a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.912+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f34440cea30 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.912+0000 7f344be00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 0x7f34380a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.912+0000 7f344be00700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 0x7f34380a3f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f345410a720 tx=0x7f343c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:30.916+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3444097970 con 0x7f34541040c0 2024-03-11T19:21:32.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.139+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f345419b770 con 0x7f34541040c0 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stdout:167503724616 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.140+0000 7f3450c00700 1 -- 172.21.15.72:0/3087349747 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f344409b030 con 0x7f34541040c0 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 msgr2=0x7f34380a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 0x7f34380a3f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f345410a720 tx=0x7f343c0073d0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 msgr2=0x7f345419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:32.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f3444005740 comp rx=0 tx=0).stop 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 shutdown_connections 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f34380a1ac0 0x7f34380a3f80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34541091a0 0x7f345419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34541040c0 0x7f345419a410 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 --2- 172.21.15.72:0/3087349747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454104a70 0x7f345419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 >> 172.21.15.72:0/3087349747 conn(0x7f34540ff950 msgr2=0x7f3454100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 shutdown_connections 2024-03-11T19:21:32.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:31.143+0000 7f345a400700 1 -- 172.21.15.72:0/3087349747 wait complete. 2024-03-11T19:21:32.636 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:32 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-11T19:21:32.636 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:32 smithi072 ceph-mon[31695]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:21:32.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:32 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-11T19:21:32.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:32 smithi106 ceph-mon[35889]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:21:32.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:32 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-11T19:21:32.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:32 smithi143 ceph-mon[36013]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:21:33.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:33 smithi072 ceph-mon[31695]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:21:33.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:33 smithi072 ceph-mon[31695]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:21:33.622 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:33 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3208433136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:21:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:33 smithi106 ceph-mon[35889]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:21:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:33 smithi106 ceph-mon[35889]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:21:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:33 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3208433136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:21:33.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:33 smithi143 ceph-mon[36013]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:21:33.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:33 smithi143 ceph-mon[36013]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:21:33.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:33 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3208433136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:21:34.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:34 smithi106 ceph-mon[35889]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:21:34.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:34 smithi072 ceph-mon[31695]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:21:34.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:34 smithi143 ceph-mon[36013]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:21:35.375 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:35 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/866832951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:21:35.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:35 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/866832951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:21:35.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:35 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/866832951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 -- 172.21.15.72:0/3814181958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4101f20 msgr2=0x7f99c4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 --2- 172.21.15.72:0/3814181958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4101f20 0x7f99c4102300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009990 tx=0x7f99b4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 -- 172.21.15.72:0/3814181958 shutdown_connections 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 --2- 172.21.15.72:0/3814181958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4106ff0 0x7f99c410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 --2- 172.21.15.72:0/3814181958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4101f20 0x7f99c4102300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 --2- 172.21.15.72:0/3814181958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c4106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 -- 172.21.15.72:0/3814181958 >> 172.21.15.72:0/3814181958 conn(0x7f99c4078dd0 msgr2=0x7f99c40791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 -- 172.21.15.72:0/3814181958 shutdown_connections 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.537+0000 7f99c9600700 1 -- 172.21.15.72:0/3814181958 wait complete. 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.538+0000 7f99c9600700 1 Processor -- start 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.538+0000 7f99c9600700 1 -- start start 2024-03-11T19:21:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 0x7f99c41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4106ff0 0x7f99c419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419d130 con 0x7f99c4106ff0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419d2a0 con 0x7f99c41028d0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c9600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419d460 con 0x7f99c4101f20 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99ba000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99ba000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:34960/0 (socket says 172.21.15.72:34960) 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99ba000700 1 -- 172.21.15.72:0/4237874624 learned_addr learned my addr 172.21.15.72:0/4237874624 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c2a00700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 0x7f99c41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.539+0000 7f99c3400700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4106ff0 0x7f99c419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 -- 172.21.15.72:0/4237874624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 msgr2=0x7f99c41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 0x7f99c41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 -- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4106ff0 msgr2=0x7f99c419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4106ff0 0x7f99c419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 -- 172.21.15.72:0/4237874624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40095f0 con 0x7f99c41028d0 2024-03-11T19:21:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99c2a00700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 0x7f99c41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99ba000700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00e970 tx=0x7f99ac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac009790 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99ac00c7e0 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac005580 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.540+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c419d740 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.541+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c41a4fa0 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.542+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c410a2b0 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.543+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f99ac00cce0 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.545+0000 7f99bbe00700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 0x7f99a80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.545+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f99ac014070 con 0x7f99c41028d0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.545+0000 7f99c2a00700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 0x7f99a80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.546+0000 7f99c2a00700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 0x7f99a80a4000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.547+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99ac096800 con 0x7f99c41028d0 2024-03-11T19:21:35.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.771+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f99c4068930 con 0x7f99c41028d0 2024-03-11T19:21:35.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.771+0000 7f99bbe00700 1 -- 172.21.15.72:0/4237874624 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f99ac099ec0 con 0x7f99c41028d0 2024-03-11T19:21:35.776 INFO:teuthology.orchestra.run.smithi072.stdout:347892351014 2024-03-11T19:21:35.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 msgr2=0x7f99a80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 0x7f99a80a4000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b400b260 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 msgr2=0x7f99c41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00e970 tx=0x7f99ac00ec80 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 shutdown_connections 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f99a80a1b40 0x7f99a80a4000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99c4106ff0 0x7f99c419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101f20 0x7f99c41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 --2- 172.21.15.72:0/4237874624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c41028d0 0x7f99c41987e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 >> 172.21.15.72:0/4237874624 conn(0x7f99c4078dd0 msgr2=0x7f99c40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 shutdown_connections 2024-03-11T19:21:35.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.774+0000 7f99c9600700 1 -- 172.21.15.72:0/4237874624 wait complete. 2024-03-11T19:21:35.854 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:36.568 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4237874624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:21:36.568 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:36 smithi072 ceph-mon[31695]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 6 op/s 2024-03-11T19:21:36.568 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:36 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2388699847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:21:36.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888135 got 146028888141 for osd.4 2024-03-11T19:21:36.631 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:36.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:36 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/4237874624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:21:36.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:36 smithi106 ceph-mon[35889]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 6 op/s 2024-03-11T19:21:36.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:36 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2388699847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:21:36.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 -- 172.21.15.72:0/829282511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678101cc0 msgr2=0x7f2678102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:36.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 --2- 172.21.15.72:0/829282511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678101cc0 0x7f2678102140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26640099c0 tx=0x7f2664009cd0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 -- 172.21.15.72:0/829282511 shutdown_connections 2024-03-11T19:21:36.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 --2- 172.21.15.72:0/829282511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678102680 0x7f267810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 --2- 172.21.15.72:0/829282511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 0x7f2678108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 --2- 172.21.15.72:0/829282511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678101cc0 0x7f2678102140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 -- 172.21.15.72:0/829282511 >> 172.21.15.72:0/829282511 conn(0x7f2678078da0 msgr2=0x7f26780791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 -- 172.21.15.72:0/829282511 shutdown_connections 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.889+0000 7f267d400700 1 -- 172.21.15.72:0/829282511 wait complete. 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.890+0000 7f267d400700 1 Processor -- start 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.890+0000 7f267d400700 1 -- start start 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 0x7f2678073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:36.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 0x7f26780749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781a65e0 con 0x7f2678102680 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781a6750 con 0x7f2678101cc0 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f267d400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781a6910 con 0x7f2678107ca0 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f2675e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f2676800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 0x7f2678073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f2675e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48378/0 (socket says 172.21.15.72:48378) 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f2675e00700 1 -- 172.21.15.72:0/866832951 learned_addr learned my addr 172.21.15.72:0/866832951 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.891+0000 7f2677200700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 0x7f26780749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 -- 172.21.15.72:0/866832951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 msgr2=0x7f26780749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 0x7f26780749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 -- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 msgr2=0x7f26780744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 -- 172.21.15.72:0/866832951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640095f0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2675e00700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f2676800700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 0x7f2678073f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f266800d750 tx=0x7f266800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668004020 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2668004830 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26781a6bf0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.892+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680055d0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.893+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26781a70f0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.894+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267810a180 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.896+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2668004ca0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.897+0000 7f266f400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 0x7f265c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.897+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f26680ccff0 con 0x7f2678101cc0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.897+0000 7f2675e00700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 0x7f265c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.898+0000 7f2675e00700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 0x7f265c0a3f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2678106bb0 tx=0x7f2664005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:36.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:34.901+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2668095e20 con 0x7f2678101cc0 2024-03-11T19:21:36.681 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.135+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2678111d00 con 0x7f2678101cc0 2024-03-11T19:21:36.681 INFO:teuthology.orchestra.run.smithi072.stdout:201863462978 2024-03-11T19:21:36.681 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.136+0000 7f266f400700 1 -- 172.21.15.72:0/866832951 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2668014090 con 0x7f2678101cc0 2024-03-11T19:21:36.681 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 msgr2=0x7f265c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 0x7f265c0a3f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2678106bb0 tx=0x7f2664005fb0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 msgr2=0x7f2678073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 0x7f2678073f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f266800d750 tx=0x7f266800da60 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 shutdown_connections 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f265c0a1ac0 0x7f265c0a3f80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2678107ca0 0x7f26780749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2678101cc0 0x7f2678073f70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 --2- 172.21.15.72:0/866832951 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2678102680 0x7f26780744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 >> 172.21.15.72:0/866832951 conn(0x7f2678078da0 msgr2=0x7f2678100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 shutdown_connections 2024-03-11T19:21:36.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:35.139+0000 7f267d400700 1 -- 172.21.15.72:0/866832951 wait complete. 2024-03-11T19:21:36.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:36 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/4237874624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:21:36.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:36 smithi143 ceph-mon[36013]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 6 op/s 2024-03-11T19:21:36.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:36 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2388699847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:21:38.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411410 got 77309411416 for osd.2 2024-03-11T19:21:38.019 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 -- 172.21.15.72:0/2326218322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd54410d5c0 msgr2=0x7fd544115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 --2- 172.21.15.72:0/2326218322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd54410d5c0 0x7fd544115190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00b0a0 tx=0x7fd53c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 -- 172.21.15.72:0/2326218322 shutdown_connections 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 --2- 172.21.15.72:0/2326218322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd54410d5c0 0x7fd544115190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 --2- 172.21.15.72:0/2326218322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 --2- 172.21.15.72:0/2326218322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd544108e90 0x7fd54410cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 -- 172.21.15.72:0/2326218322 >> 172.21.15.72:0/2326218322 conn(0x7fd544075440 msgr2=0x7fd544077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 -- 172.21.15.72:0/2326218322 shutdown_connections 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.001+0000 7fd54c400700 1 -- 172.21.15.72:0/2326218322 wait complete. 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.002+0000 7fd54c400700 1 Processor -- start 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.002+0000 7fd54c400700 1 -- start start 2024-03-11T19:21:38.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 0x7fd5441a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd54410d5c0 0x7fd54419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419ee90 con 0x7fd544108e90 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419f000 con 0x7fd5441084e0 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54c400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419f1c0 con 0x7fd54410d5c0 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 0x7fd5441a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:34980/0 (socket says 172.21.15.72:34980) 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 -- 172.21.15.72:0/2388699847 learned_addr learned my addr 172.21.15.72:0/2388699847 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd54a400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd54410d5c0 0x7fd54419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 -- 172.21.15.72:0/2388699847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd54410d5c0 msgr2=0x7fd54419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd54410d5c0 0x7fd54419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 -- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 msgr2=0x7fd5441a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:38.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 0x7fd5441a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549a00700 1 -- 172.21.15.72:0/2388699847 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c009d00 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.003+0000 7fd549000700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 0x7fd5441a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd549a00700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9a0 tx=0x7fd53800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538009790 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54419f4a0 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd53800cbd0 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5441ab5e0 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.004+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538018680 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.005+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5441108e0 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.007+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd538005070 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.009+0000 7fd542000700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 0x7fd52c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.009+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fd538014070 con 0x7fd5441084e0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.009+0000 7fd549000700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 0x7fd52c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:38.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.010+0000 7fd549000700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 0x7fd52c0a4000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd54419fea0 tx=0x7fd530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:38.134 INFO:teuthology.orchestra.run.smithi072.stdout:223338299452 2024-03-11T19:21:38.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.011+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5380968b0 con 0x7fd5441084e0 2024-03-11T19:21:38.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.232+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd544068930 con 0x7fd5441084e0 2024-03-11T19:21:38.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.233+0000 7fd542000700 1 -- 172.21.15.72:0/2388699847 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd538099f70 con 0x7fd5441084e0 2024-03-11T19:21:38.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 msgr2=0x7fd52c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 0x7fd52c0a4000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd54419fea0 tx=0x7fd530009260 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 msgr2=0x7fd5441a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9a0 tx=0x7fd53800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 shutdown_connections 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd52c0a1b40 0x7fd52c0a4000 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd54410d5c0 0x7fd54419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5441084e0 0x7fd5441a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 --2- 172.21.15.72:0/2388699847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd544108e90 0x7fd5441a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 >> 172.21.15.72:0/2388699847 conn(0x7fd544075440 msgr2=0x7fd544075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 shutdown_connections 2024-03-11T19:21:38.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:36.236+0000 7fd54c400700 1 -- 172.21.15.72:0/2388699847 wait complete. 2024-03-11T19:21:39.374 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299445 got 223338299452 for osd.7 2024-03-11T19:21:39.374 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:39 smithi072 ceph-mon[31695]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-11T19:21:39.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:39 smithi143 ceph-mon[36013]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-11T19:21:39.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:39 smithi106 ceph-mon[35889]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-03-11T19:21:41.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:41 smithi072 ceph-mon[31695]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 464 B/s wr, 7 op/s 2024-03-11T19:21:41.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:41 smithi143 ceph-mon[36013]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 464 B/s wr, 7 op/s 2024-03-11T19:21:41.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724609 got 167503724616 for osd.5 2024-03-11T19:21:41.615 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:41 smithi106 ceph-mon[35889]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 464 B/s wr, 7 op/s 2024-03-11T19:21:43.402 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:43 smithi072 ceph-mon[31695]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-03-11T19:21:43.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:43 smithi143 ceph-mon[36013]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-03-11T19:21:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:43 smithi106 ceph-mon[35889]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-03-11T19:21:44.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 -- 172.21.15.72:0/2040626864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597010e550 msgr2=0x7f5970110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 --2- 172.21.15.72:0/2040626864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597010e550 0x7f5970110940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5968009f90 tx=0x7f596800b330 comp rx=0 tx=0).stop 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 -- 172.21.15.72:0/2040626864 shutdown_connections 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 --2- 172.21.15.72:0/2040626864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597010e550 0x7f5970110940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 --2- 172.21.15.72:0/2040626864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970073f90 0x7f5970074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 --2- 172.21.15.72:0/2040626864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59700748b0 0x7f597010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 -- 172.21.15.72:0/2040626864 >> 172.21.15.72:0/2040626864 conn(0x7f59700fd740 msgr2=0x7f59700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 -- 172.21.15.72:0/2040626864 shutdown_connections 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.420+0000 7f5978400700 1 -- 172.21.15.72:0/2040626864 wait complete. 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 Processor -- start 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 -- start start 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 0x7f597019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597010e550 0x7f597019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597019f440 con 0x7f597010e550 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597019f5b0 con 0x7f59700748b0 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.422+0000 7f5978400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597019f770 con 0x7f5970073f90 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:58650/0 (socket says 172.21.15.72:58650) 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975a00700 1 -- 172.21.15.72:0/1558503892 learned_addr learned my addr 172.21.15.72:0/1558503892 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 0x7f597019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5976400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597010e550 0x7f597019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 -- 172.21.15.72:0/1558503892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 msgr2=0x7f597019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 -- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597010e550 msgr2=0x7f597019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597010e550 0x7f597019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975000700 1 -- 172.21.15.72:0/1558503892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640095f0 con 0x7f59700748b0 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.423+0000 7f5975a00700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f5975000700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 0x7f597019aaa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000c9c0 con 0x7f59700748b0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5960010430 con 0x7f59700748b0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960018680 con 0x7f59700748b0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5968009c40 con 0x7f59700748b0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.424+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59701a70e0 con 0x7f59700748b0 2024-03-11T19:21:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.425+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970109760 con 0x7f59700748b0 2024-03-11T19:21:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.427+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5960010910 con 0x7f59700748b0 2024-03-11T19:21:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.429+0000 7f596e000700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 0x7f59580a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.429+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f5960014070 con 0x7f59700748b0 2024-03-11T19:21:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.429+0000 7f5975a00700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 0x7f59580a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.430+0000 7f5975a00700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 0x7f59580a4000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5964005d10 tx=0x7f5964009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.431+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5960096660 con 0x7f59700748b0 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.282+0000 7fe23ba00700 1 -- 172.21.15.72:0/3866208007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 msgr2=0x7fe23410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.282+0000 7fe23ba00700 1 --2- 172.21.15.72:0/3866208007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23410e130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098f0 tx=0x7fe220009c00 comp rx=0 tx=0).stop 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 -- 172.21.15.72:0/3866208007 shutdown_connections 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 --2- 172.21.15.72:0/3866208007 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 0x7fe234110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 --2- 172.21.15.72:0/3866208007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 0x7fe234074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 --2- 172.21.15.72:0/3866208007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23410e130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 -- 172.21.15.72:0/3866208007 >> 172.21.15.72:0/3866208007 conn(0x7fe234078dd0 msgr2=0x7fe2340791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 -- 172.21.15.72:0/3866208007 shutdown_connections 2024-03-11T19:21:44.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.283+0000 7fe23ba00700 1 -- 172.21.15.72:0/3866208007 wait complete. 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 Processor -- start 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 -- start start 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 0x7fe23419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 0x7fe23419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419f3d0 con 0x7fe23410e670 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419f540 con 0x7fe234073ea0 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.284+0000 7fe23ba00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419f700 con 0x7fe2340747c0 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:58632/0 (socket says 172.21.15.72:58632) 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 -- 172.21.15.72:0/1005392684 learned_addr learned my addr 172.21.15.72:0/1005392684 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe239000700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 0x7fe23419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe239a00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 0x7fe23419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 -- 172.21.15.72:0/1005392684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 msgr2=0x7fe23419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 0x7fe23419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 -- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 msgr2=0x7fe23419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 0x7fe23419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.285+0000 7fe233e00700 1 -- 172.21.15.72:0/1005392684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009520 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe233e00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe220005e90 tx=0x7fe22000fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22001c070 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe22000b560 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23419f980 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220003c60 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.286+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2341a7240 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.287+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe234109880 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.289+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe22000b6d0 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.290+0000 7fe231600700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 0x7fe21c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.290+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fe2200ce300 con 0x7fe2340747c0 2024-03-11T19:21:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.290+0000 7fe239000700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 0x7fe21c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:44.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.291+0000 7fe239000700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 0x7fe21c0a3f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe228009c60 tx=0x7fe228009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:44.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.294+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2200971c0 con 0x7fe2340747c0 2024-03-11T19:21:44.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.521+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe23419b850 con 0x7fe2340747c0 2024-03-11T19:21:44.613 INFO:teuthology.orchestra.run.smithi072.stdout:55834574945 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.522+0000 7fe231600700 1 -- 172.21.15.72:0/1005392684 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe22009a880 con 0x7fe2340747c0 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 msgr2=0x7fe21c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 0x7fe21c0a3f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe228009c60 tx=0x7fe228009260 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 msgr2=0x7fe23419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe220005e90 tx=0x7fe22000fa90 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 shutdown_connections 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fe21c0a1ac0 0x7fe21c0a3f80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe23410e670 0x7fe23419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe234073ea0 0x7fe23419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 --2- 172.21.15.72:0/1005392684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2340747c0 0x7fe23419aa80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 >> 172.21.15.72:0/1005392684 conn(0x7fe234078dd0 msgr2=0x7fe2341001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:44.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 shutdown_connections 2024-03-11T19:21:44.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.525+0000 7fe23ba00700 1 -- 172.21.15.72:0/1005392684 wait complete. 2024-03-11T19:21:44.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.666+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f597019b920 con 0x7f59700748b0 2024-03-11T19:21:44.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.667+0000 7f596e000700 1 -- 172.21.15.72:0/1558503892 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59600105a0 con 0x7f59700748b0 2024-03-11T19:21:44.670 INFO:teuthology.orchestra.run.smithi072.stdout:34359738470 2024-03-11T19:21:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 msgr2=0x7f59580a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 0x7f59580a4000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5964005d10 tx=0x7f5964009310 comp rx=0 tx=0).stop 2024-03-11T19:21:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 msgr2=0x7f597019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 0x7f597019aaa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000ed00 comp rx=0 tx=0).stop 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 shutdown_connections 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f59580a1b40 0x7f59580a4000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597010e550 0x7f597019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5970073f90 0x7f597019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 --2- 172.21.15.72:0/1558503892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59700748b0 0x7f597019aaa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 >> 172.21.15.72:0/1558503892 conn(0x7f59700fd740 msgr2=0x7f59700fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 shutdown_connections 2024-03-11T19:21:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:44.670+0000 7f5978400700 1 -- 172.21.15.72:0/1558503892 wait complete. 2024-03-11T19:21:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:45 smithi143 ceph-mon[36013]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:21:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:45 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1005392684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:21:45.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:45 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1558503892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:21:45.445 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:45 smithi072 ceph-mon[31695]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:21:45.445 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:45 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1005392684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:21:45.445 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:45 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1558503892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:21:45.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 -- 172.21.15.72:0/4132361891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 msgr2=0x7f02dc1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 --2- 172.21.15.72:0/4132361891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc1006e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009990 tx=0x7f02c8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 -- 172.21.15.72:0/4132361891 shutdown_connections 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 --2- 172.21.15.72:0/4132361891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 0x7f02dc10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 --2- 172.21.15.72:0/4132361891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 0x7f02dc1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 --2- 172.21.15.72:0/4132361891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc1006e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 -- 172.21.15.72:0/4132361891 >> 172.21.15.72:0/4132361891 conn(0x7f02dc077170 msgr2=0x7f02dc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 -- 172.21.15.72:0/4132361891 shutdown_connections 2024-03-11T19:21:45.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.484+0000 7f02e4400700 1 -- 172.21.15.72:0/4132361891 wait complete. 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.485+0000 7f02e4400700 1 Processor -- start 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.485+0000 7f02e4400700 1 -- start start 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 0x7f02dc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 0x7f02dc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19d2e0 con 0x7f02dc100260 2024-03-11T19:21:45.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19d450 con 0x7f02dc100c20 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e4400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19d610 con 0x7f02dc106ce0 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41038/0 (socket says 172.21.15.72:41038) 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1a00700 1 -- 172.21.15.72:0/3884187630 learned_addr learned my addr 172.21.15.72:0/3884187630 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1000700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 0x7f02dc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:45.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e2400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 0x7f02dc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1a00700 1 -- 172.21.15.72:0/3884187630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 msgr2=0x7f02dc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.486+0000 7f02e1a00700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 0x7f02dc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e1a00700 1 -- 172.21.15.72:0/3884187630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 msgr2=0x7f02dc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e1a00700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 0x7f02dc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e1a00700 1 -- 172.21.15.72:0/3884187630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80095f0 con 0x7f02dc100260 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e1a00700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9a0 tx=0x7f02d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0009790 con 0x7f02dc100260 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02d0010430 con 0x7f02dc100260 2024-03-11T19:21:45.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0018680 con 0x7f02dc100260 2024-03-11T19:21:45.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc19d8f0 con 0x7f02dc100260 2024-03-11T19:21:45.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.487+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc1a5150 con 0x7f02dc100260 2024-03-11T19:21:45.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.489+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc10a2d0 con 0x7f02dc100260 2024-03-11T19:21:45.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.490+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f02d0010910 con 0x7f02dc100260 2024-03-11T19:21:45.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.492+0000 7f02da000700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 0x7f02c40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:45.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.492+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f02d0014070 con 0x7f02dc100260 2024-03-11T19:21:45.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.492+0000 7f02e1000700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 0x7f02c40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:45.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.493+0000 7f02e1000700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 0x7f02c40a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0ff7a0 tx=0x7f02c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:45.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.496+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f02d0052d20 con 0x7f02dc100260 2024-03-11T19:21:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:45 smithi106 ceph-mon[35889]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:21:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:45 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1005392684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:21:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:45 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1558503892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:21:45.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.719+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f02dc19dae0 con 0x7f02dc100260 2024-03-11T19:21:45.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.720+0000 7f02da000700 1 -- 172.21.15.72:0/3884187630 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f02d00967c0 con 0x7f02dc100260 2024-03-11T19:21:45.724 INFO:teuthology.orchestra.run.smithi072.stdout:313532612651 2024-03-11T19:21:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 msgr2=0x7f02c40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 0x7f02c40a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0ff7a0 tx=0x7f02c8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 msgr2=0x7f02dc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9a0 tx=0x7f02d000ecb0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 shutdown_connections 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f02c40a1ac0 0x7f02c40a3f80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02dc106ce0 0x7f02dc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02dc100260 0x7f02dc198400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 --2- 172.21.15.72:0/3884187630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc100c20 0x7f02dc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 >> 172.21.15.72:0/3884187630 conn(0x7f02dc077170 msgr2=0x7f02dc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 shutdown_connections 2024-03-11T19:21:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:45.723+0000 7f02e4400700 1 -- 172.21.15.72:0/3884187630 wait complete. 2024-03-11T19:21:46.335 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3884187630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:21:46.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:46 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3884187630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:21:46.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:46 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3884187630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:21:46.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738463 got 34359738470 for osd.0 2024-03-11T19:21:46.943 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:47.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574945 for osd.1 2024-03-11T19:21:47.383 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:47.481 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:47 smithi072 ceph-mon[31695]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:21:47.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:47 smithi106 ceph-mon[35889]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:21:47.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 -- 172.21.15.72:0/3953963103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c108e90 msgr2=0x7f805c10cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:47.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 --2- 172.21.15.72:0/3953963103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c108e90 0x7f805c10cee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8044009990 tx=0x7f8044009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 -- 172.21.15.72:0/3953963103 shutdown_connections 2024-03-11T19:21:47.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 --2- 172.21.15.72:0/3953963103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 --2- 172.21.15.72:0/3953963103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c1084e0 0x7f805c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 --2- 172.21.15.72:0/3953963103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c108e90 0x7f805c10cee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 -- 172.21.15.72:0/3953963103 >> 172.21.15.72:0/3953963103 conn(0x7f805c075440 msgr2=0x7f805c077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 -- 172.21.15.72:0/3953963103 shutdown_connections 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.643+0000 7f8062e00700 1 -- 172.21.15.72:0/3953963103 wait complete. 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.644+0000 7f8062e00700 1 Processor -- start 2024-03-11T19:21:47.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.644+0000 7f8062e00700 1 -- start start 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c1084e0 0x7f805c1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c19ec90 con 0x7f805c1084e0 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c19edd0 con 0x7f805c108e90 2024-03-11T19:21:47.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8062e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c19ef90 con 0x7f805c10d5c0 2024-03-11T19:21:47.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8053400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:47.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f805be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c1084e0 0x7f805c1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:47.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8053400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:55386/0 (socket says 172.21.15.72:55386) 2024-03-11T19:21:47.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8053400700 1 -- 172.21.15.72:0/167352997 learned_addr learned my addr 172.21.15.72:0/167352997 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:47.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.645+0000 7f8060e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 -- 172.21.15.72:0/167352997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 msgr2=0x7f805c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 -- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c1084e0 msgr2=0x7f805c1a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c1084e0 0x7f805c1a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 -- 172.21.15.72:0/167352997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440095f0 con 0x7f805c108e90 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8060e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8053400700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8044000c00 tx=0x7f80440057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804401d070 con 0x7f805c108e90 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c19f210 con 0x7f805c108e90 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8044021410 con 0x7f805c108e90 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044017650 con 0x7f805c108e90 2024-03-11T19:21:47.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.646+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c1ab4c0 con 0x7f805c108e90 2024-03-11T19:21:47.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.647+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c1108e0 con 0x7f805c108e90 2024-03-11T19:21:47.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.649+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8044021580 con 0x7f805c108e90 2024-03-11T19:21:47.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.651+0000 7f8059600700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 0x7f80400a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:47.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.651+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f80440cf110 con 0x7f805c108e90 2024-03-11T19:21:47.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.651+0000 7f805be00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 0x7f80400a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:47.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.652+0000 7f805be00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 0x7f80400a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f804c00f540 tx=0x7f804c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:47.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.653+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8044098090 con 0x7f805c108e90 2024-03-11T19:21:47.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:47 smithi143 ceph-mon[36013]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:21:47.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.878+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f805c19fb10 con 0x7f805c108e90 2024-03-11T19:21:47.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.879+0000 7f8059600700 1 -- 172.21.15.72:0/167352997 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80440a0e60 con 0x7f805c108e90 2024-03-11T19:21:47.882 INFO:teuthology.orchestra.run.smithi072.stdout:292057776178 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 msgr2=0x7f80400a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 0x7f80400a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f804c00f540 tx=0x7f804c00b040 comp rx=0 tx=0).stop 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 msgr2=0x7f805c1a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8044000c00 tx=0x7f80440057c0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 shutdown_connections 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80400a1ac0 0x7f80400a3f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f805c10d5c0 0x7f805c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f805c1084e0 0x7f805c1a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.881+0000 7f8062e00700 1 --2- 172.21.15.72:0/167352997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805c108e90 0x7f805c1a4960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:47.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.882+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 >> 172.21.15.72:0/167352997 conn(0x7f805c075440 msgr2=0x7f805c075fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:47.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.882+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 shutdown_connections 2024-03-11T19:21:47.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.882+0000 7f8062e00700 1 -- 172.21.15.72:0/167352997 wait complete. 2024-03-11T19:21:48.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351008 got 347892351014 for osd.11 2024-03-11T19:21:48.423 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:48.533 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/167352997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:21:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/167352997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:21:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/167352997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:21:49.393 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462970 got 201863462978 for osd.6 2024-03-11T19:21:49.393 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:49.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:49 smithi106 ceph-mon[35889]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:49.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3691847292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:21:49.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:49 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/834256283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:21:49.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:49 smithi072 ceph-mon[31695]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:49.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3691847292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:21:49.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:49 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/834256283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:21:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:49 smithi143 ceph-mon[36013]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3691847292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:21:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:49 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/834256283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:21:49.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776171 got 292057776178 for osd.9 2024-03-11T19:21:49.857 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:49.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 -- 172.21.15.72:0/2373153334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30101f20 msgr2=0x7fdf30102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 --2- 172.21.15.72:0/2373153334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30101f20 0x7fdf30102300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf200098f0 tx=0x7fdf20009c00 comp rx=0 tx=0).stop 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 -- 172.21.15.72:0/2373153334 shutdown_connections 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 --2- 172.21.15.72:0/2373153334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30106ff0 0x7fdf3010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 --2- 172.21.15.72:0/2373153334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30101f20 0x7fdf30102300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 --2- 172.21.15.72:0/2373153334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf301028d0 0x7fdf30106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 -- 172.21.15.72:0/2373153334 >> 172.21.15.72:0/2373153334 conn(0x7fdf30078dd0 msgr2=0x7fdf300791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 -- 172.21.15.72:0/2373153334 shutdown_connections 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.967+0000 7fdf37000700 1 -- 172.21.15.72:0/2373153334 wait complete. 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 Processor -- start 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 -- start start 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf30101f20 0x7fdf30198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:49.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 0x7fdf301987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.968+0000 7fdf37000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d140 con 0x7fdf30106ff0 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf37000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d2b0 con 0x7fdf301028d0 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf37000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3019d470 con 0x7fdf30101f20 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf35000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 0x7fdf301987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf35000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41064/0 (socket says 172.21.15.72:41064) 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf35000700 1 -- 172.21.15.72:0/3691847292 learned_addr learned my addr 172.21.15.72:0/3691847292 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2fe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf30101f20 0x7fdf30198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 -- 172.21.15.72:0/3691847292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf30101f20 msgr2=0x7fdf30198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf30101f20 0x7fdf30198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 -- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 msgr2=0x7fdf3019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf2f400700 1 -- 172.21.15.72:0/3691847292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0095f0 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.969+0000 7fdf35000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf2f400700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 0x7fdf301987a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00cbd0 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c004500 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf20009520 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c010430 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.970+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf301a4de0 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.971+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3010a2b0 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.973+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdf1c003680 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.975+0000 7fdf2cc00700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 0x7fdf140a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.975+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fdf1c014070 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.975+0000 7fdf2fe00700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 0x7fdf140a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.976+0000 7fdf2fe00700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 0x7fdf140a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf200098c0 tx=0x7fdf20019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:47.976+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf1c09a660 con 0x7fdf301028d0 2024-03-11T19:21:49.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.203+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdf301995c0 con 0x7fdf301028d0 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.204+0000 7fdf2cc00700 1 -- 172.21.15.72:0/3691847292 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf1c09a1f0 con 0x7fdf301028d0 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stdout:111669149782 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 msgr2=0x7fdf140a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 0x7fdf140a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf200098c0 tx=0x7fdf20019040 comp rx=0 tx=0).stop 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 msgr2=0x7fdf301987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 0x7fdf301987a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c00ee70 comp rx=0 tx=0).stop 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 shutdown_connections 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdf140a1ac0 0x7fdf140a3f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf30106ff0 0x7fdf3019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf30101f20 0x7fdf30198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 --2- 172.21.15.72:0/3691847292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf301028d0 0x7fdf301987a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:49.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.206+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 >> 172.21.15.72:0/3691847292 conn(0x7fdf30078dd0 msgr2=0x7fdf301081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:49.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.207+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 shutdown_connections 2024-03-11T19:21:49.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.207+0000 7fdf37000700 1 -- 172.21.15.72:0/3691847292 wait complete. 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 -- 172.21.15.72:0/400518865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 msgr2=0x7f990810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 --2- 172.21.15.72:0/400518865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990810dfb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009990 tx=0x7f98f4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 -- 172.21.15.72:0/400518865 shutdown_connections 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 --2- 172.21.15.72:0/400518865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f9908110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 --2- 172.21.15.72:0/400518865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f9908074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 --2- 172.21.15.72:0/400518865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990810dfb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 -- 172.21.15.72:0/400518865 >> 172.21.15.72:0/400518865 conn(0x7f99080fd720 msgr2=0x7f99080ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 -- 172.21.15.72:0/400518865 shutdown_connections 2024-03-11T19:21:50.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.692+0000 7f990e400700 1 -- 172.21.15.72:0/400518865 wait complete. 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.693+0000 7f990e400700 1 Processor -- start 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 -- start start 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819f350 con 0x7f9908074960 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819f4c0 con 0x7f990810e4f0 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f990e400700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819f680 con 0x7f9908074040 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9907400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9907e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9907400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:58716/0 (socket says 172.21.15.72:58716) 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9907e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:55418/0 (socket says 172.21.15.72:55418) 2024-03-11T19:21:50.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9907400700 1 -- 172.21.15.72:0/834256283 learned_addr learned my addr 172.21.15.72:0/834256283 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.694+0000 7f9906a00700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 -- 172.21.15.72:0/834256283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 msgr2=0x7f990819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 -- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 msgr2=0x7f990819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 -- 172.21.15.72:0/834256283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8009520 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f9907e00700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f990000ef90 tx=0x7f990000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.695+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900018070 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.696+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f990000f040 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.696+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f40095f0 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.696+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000cd40 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.696+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99081a6ff0 con 0x7f990810e4f0 2024-03-11T19:21:50.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.697+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9908109700 con 0x7f990810e4f0 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.699+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9900003a00 con 0x7f990810e4f0 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.700+0000 7f98ffe00700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 0x7f98ec0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.700+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f99000cd1b0 con 0x7f990810e4f0 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.700+0000 7f9907400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 0x7f98ec0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.701+0000 7f9907400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 0x7f98ec0a4000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009fd0 tx=0x7f98f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:50.312 INFO:teuthology.orchestra.run.smithi072.stdout:257698037817 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.704+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9900096070 con 0x7f990810e4f0 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.928+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f990819b7d0 con 0x7f990810e4f0 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.929+0000 7f98ffe00700 1 -- 172.21.15.72:0/834256283 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9900099730 con 0x7f990810e4f0 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 msgr2=0x7f98ec0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 0x7f98ec0a4000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009fd0 tx=0x7f98f8009260 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 msgr2=0x7f990819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f990000ef90 tx=0x7f990000c3a0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 shutdown_connections 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f98ec0a1b40 0x7f98ec0a4000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990810e4f0 0x7f990819ed40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9908074040 0x7f990819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 --2- 172.21.15.72:0/834256283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9908074960 0x7f990819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 >> 172.21.15.72:0/834256283 conn(0x7f99080fd720 msgr2=0x7f99080fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:50.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.932+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 shutdown_connections 2024-03-11T19:21:50.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:48.933+0000 7f990e400700 1 -- 172.21.15.72:0/834256283 wait complete. 2024-03-11T19:21:51.552 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:51 smithi072 ceph-mon[31695]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:51.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037810 got 257698037817 for osd.8 2024-03-11T19:21:51.622 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:51.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:51 smithi106 ceph-mon[35889]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:51.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:51 smithi143 ceph-mon[36013]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:51.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612644 got 313532612651 for osd.10 2024-03-11T19:21:51.759 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:52.310 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149774 got 111669149782 for osd.3 2024-03-11T19:21:52.311 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:52.311 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-11T19:21:52.311 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:21:52.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:21:52.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:21:52.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:52 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:21:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:21:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:52 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:21:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:21:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:52 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:21:52.769 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:53.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:53 smithi106 ceph-mon[35889]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:53.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:53 smithi072 ceph-mon[31695]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:53 smithi143 ceph-mon[36013]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 -- 172.21.15.72:0/3400189613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c010d5c0 msgr2=0x7f80c0115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 --2- 172.21.15.72:0/3400189613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c010d5c0 0x7f80c0115190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b0a0 tx=0x7f80b800b3b0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 -- 172.21.15.72:0/3400189613 shutdown_connections 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 --2- 172.21.15.72:0/3400189613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c010d5c0 0x7f80c0115190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 --2- 172.21.15.72:0/3400189613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c01084e0 0x7f80c01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 --2- 172.21.15.72:0/3400189613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c0108e90 0x7f80c010cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 -- 172.21.15.72:0/3400189613 >> 172.21.15.72:0/3400189613 conn(0x7f80c0075440 msgr2=0x7f80c0077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:55.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 -- 172.21.15.72:0/3400189613 shutdown_connections 2024-03-11T19:21:55.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.109+0000 7f80c7000700 1 -- 172.21.15.72:0/3400189613 wait complete. 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.110+0000 7f80c7000700 1 Processor -- start 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 -- start start 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 0x7f80c01a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c0108e90 0x7f80c01a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 0x7f80c019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c019ee90 con 0x7f80c010d5c0 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c019f000 con 0x7f80c0108e90 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c7000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c019f1c0 con 0x7f80c01084e0 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 0x7f80c01a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:55.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 0x7f80c01a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:57786/0 (socket says 172.21.15.72:57786) 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80c5000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 0x7f80c019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80bfe00700 1 -- 172.21.15.72:0/2870802239 learned_addr learned my addr 172.21.15.72:0/2870802239 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.111+0000 7f80bf400700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c0108e90 0x7f80c01a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 -- 172.21.15.72:0/2870802239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 msgr2=0x7f80c01a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 0x7f80c01a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 -- 172.21.15.72:0/2870802239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c0108e90 msgr2=0x7f80c01a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c0108e90 0x7f80c01a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 -- 172.21.15.72:0/2870802239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac009520 con 0x7f80c010d5c0 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c5000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 0x7f80c019e6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b0a0 tx=0x7f80b800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800e040 con 0x7f80c010d5c0 2024-03-11T19:21:55.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b80079d0 con 0x7f80c010d5c0 2024-03-11T19:21:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8005040 con 0x7f80c010d5c0 2024-03-11T19:21:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.112+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b8009d00 con 0x7f80c010d5c0 2024-03-11T19:21:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.113+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c01ab3c0 con 0x7f80c010d5c0 2024-03-11T19:21:55.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.114+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c01099c0 con 0x7f80c010d5c0 2024-03-11T19:21:55.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.117+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f80b801d600 con 0x7f80c010d5c0 2024-03-11T19:21:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.119+0000 7f80bcc00700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 0x7f80a40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.119+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f80b80cf1b0 con 0x7f80c010d5c0 2024-03-11T19:21:55.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.119+0000 7f80bfe00700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 0x7f80a40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:55.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.120+0000 7f80bfe00700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 0x7f80a40a3f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009d20 tx=0x7f80b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:55.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.122+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80b8098070 con 0x7f80c010d5c0 2024-03-11T19:21:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.338+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80c0118390 con 0x7f80a40a1ac0 2024-03-11T19:21:55.343 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:55 smithi072 ceph-mon[31695]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:55.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.352+0000 7f80bcc00700 1 -- 172.21.15.72:0/2870802239 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340795 (secure 0 0 0) 0x7f80c0118390 con 0x7f80a40a1ac0 2024-03-11T19:21:55.359 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.360+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 msgr2=0x7f80a40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.360+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 0x7f80a40a3f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009d20 tx=0x7f80b0009260 comp rx=0 tx=0).stop 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.360+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 msgr2=0x7f80c019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.360+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 0x7f80c019e6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b0a0 tx=0x7f80b800bdd0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 shutdown_connections 2024-03-11T19:21:55.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f80a40a1ac0 0x7f80a40a3f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80c010d5c0 0x7f80c019e6e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80c01084e0 0x7f80c01a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 --2- 172.21.15.72:0/2870802239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c0108e90 0x7f80c01a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 >> 172.21.15.72:0/2870802239 conn(0x7f80c0075440 msgr2=0x7f80c0075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 shutdown_connections 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:55.361+0000 7f80c7000700 1 -- 172.21.15.72:0/2870802239 wait complete. 2024-03-11T19:21:55.367 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-03-11T19:21:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:55 smithi106 ceph-mon[35889]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:55 smithi143 ceph-mon[36013]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:21:56.351 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-03-11T19:21:54.113442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":744,"num_read_kb":493,"num_write":353,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370084,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571740,"statfs":{"total":1151940427776,"available":1151561461760,"internally_reserved":0,"allocated":12836864,"data_stored":8225492,"data_compressed":431934,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005955"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481428+0000","last_change":"2024-03-11T19:19:42.386329+0000","last_active":"2024-03-11T19:21:32.481428+0000","last_peered":"2024-03-11T19:21:32.481428+0000","last_clean":"2024-03-11T19:21:32.481428+0000","last_became_active":"2024-03-11T19:19:42.386041+0000","last_became_peered":"2024-03-11T19:19:42.386041+0000","last_unstale":"2024-03-11T19:21:32.481428+0000","last_undegraded":"2024-03-11T19:21:32.481428+0000","last_fullsized":"2024-03-11T19:21:32.481428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:05:38.258840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481854+0000","last_change":"2024-03-11T19:19:42.381508+0000","last_active":"2024-03-11T19:21:32.481854+0000","last_peered":"2024-03-11T19:21:32.481854+0000","last_clean":"2024-03-11T19:21:32.481854+0000","last_became_active":"2024-03-11T19:19:42.378340+0000","last_became_peered":"2024-03-11T19:19:42.378340+0000","last_unstale":"2024-03-11T19:21:32.481854+0000","last_undegraded":"2024-03-11T19:21:32.481854+0000","last_fullsized":"2024-03-11T19:21:32.481854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:31:06.472488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481293+0000","last_change":"2024-03-11T19:19:42.377904+0000","last_active":"2024-03-11T19:21:32.481293+0000","last_peered":"2024-03-11T19:21:32.481293+0000","last_clean":"2024-03-11T19:21:32.481293+0000","last_became_active":"2024-03-11T19:19:42.376279+0000","last_became_peered":"2024-03-11T19:19:42.376279+0000","last_unstale":"2024-03-11T19:21:32.481293+0000","last_undegraded":"2024-03-11T19:21:32.481293+0000","last_fullsized":"2024-03-11T19:21:32.481293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:52.685543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.325869+0000","last_change":"2024-03-11T19:21:32.324870+0000","last_active":"2024-03-11T19:21:32.325869+0000","last_peered":"2024-03-11T19:21:32.325869+0000","last_clean":"2024-03-11T19:21:32.325869+0000","last_became_active":"2024-03-11T19:21:32.323768+0000","last_became_peered":"2024-03-11T19:21:32.323768+0000","last_unstale":"2024-03-11T19:21:32.325869+0000","last_undegraded":"2024-03-11T19:21:32.325869+0000","last_fullsized":"2024-03-11T19:21:32.325869+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:18:47.489312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322158+0000","last_change":"2024-03-11T19:19:42.380979+0000","last_active":"2024-03-11T19:21:32.322158+0000","last_peered":"2024-03-11T19:21:32.322158+0000","last_clean":"2024-03-11T19:21:32.322158+0000","last_became_active":"2024-03-11T19:19:42.380653+0000","last_became_peered":"2024-03-11T19:19:42.380653+0000","last_unstale":"2024-03-11T19:21:32.322158+0000","last_undegraded":"2024-03-11T19:21:32.322158+0000","last_fullsized":"2024-03-11T19:21:32.322158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:39.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313025+0000","last_change":"2024-03-11T19:19:42.378182+0000","last_active":"2024-03-11T19:21:32.313025+0000","last_peered":"2024-03-11T19:21:32.313025+0000","last_clean":"2024-03-11T19:21:32.313025+0000","last_became_active":"2024-03-11T19:19:42.377810+0000","last_became_peered":"2024-03-11T19:19:42.377810+0000","last_unstale":"2024-03-11T19:21:32.313025+0000","last_undegraded":"2024-03-11T19:21:32.313025+0000","last_fullsized":"2024-03-11T19:21:32.313025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:37:20.498630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480620+0000","last_change":"2024-03-11T19:19:42.370872+0000","last_active":"2024-03-11T19:21:32.480620+0000","last_peered":"2024-03-11T19:21:32.480620+0000","last_clean":"2024-03-11T19:21:32.480620+0000","last_became_active":"2024-03-11T19:19:42.370543+0000","last_became_peered":"2024-03-11T19:19:42.370543+0000","last_unstale":"2024-03-11T19:21:32.480620+0000","last_undegraded":"2024-03-11T19:21:32.480620+0000","last_fullsized":"2024-03-11T19:21:32.480620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:48.483035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317106+0000","last_change":"2024-03-11T19:20:31.330881+0000","last_active":"2024-03-11T19:21:32.317106+0000","last_peered":"2024-03-11T19:21:32.317106+0000","last_clean":"2024-03-11T19:21:32.317106+0000","last_became_active":"2024-03-11T19:20:31.328780+0000","last_became_peered":"2024-03-11T19:20:31.328780+0000","last_unstale":"2024-03-11T19:21:32.317106+0000","last_undegraded":"2024-03-11T19:21:32.317106+0000","last_fullsized":"2024-03-11T19:21:32.317106+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:18.157119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481834+0000","last_change":"2024-03-11T19:19:42.368501+0000","last_active":"2024-03-11T19:21:32.481834+0000","last_peered":"2024-03-11T19:21:32.481834+0000","last_clean":"2024-03-11T19:21:32.481834+0000","last_became_active":"2024-03-11T19:19:42.368155+0000","last_became_peered":"2024-03-11T19:19:42.368155+0000","last_unstale":"2024-03-11T19:21:32.481834+0000","last_undegraded":"2024-03-11T19:21:32.481834+0000","last_fullsized":"2024-03-11T19:21:32.481834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:55:59.244376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313368+0000","last_change":"2024-03-11T19:19:42.365113+0000","last_active":"2024-03-11T19:21:32.313368+0000","last_peered":"2024-03-11T19:21:32.313368+0000","last_clean":"2024-03-11T19:21:32.313368+0000","last_became_active":"2024-03-11T19:19:42.364710+0000","last_became_peered":"2024-03-11T19:19:42.364710+0000","last_unstale":"2024-03-11T19:21:32.313368+0000","last_undegraded":"2024-03-11T19:21:32.313368+0000","last_fullsized":"2024-03-11T19:21:32.313368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:28.999688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481609+0000","last_change":"2024-03-11T19:19:42.365977+0000","last_active":"2024-03-11T19:21:32.481609+0000","last_peered":"2024-03-11T19:21:32.481609+0000","last_clean":"2024-03-11T19:21:32.481609+0000","last_became_active":"2024-03-11T19:19:42.365220+0000","last_became_peered":"2024-03-11T19:19:42.365220+0000","last_unstale":"2024-03-11T19:21:32.481609+0000","last_undegraded":"2024-03-11T19:21:32.481609+0000","last_fullsized":"2024-03-11T19:21:32.481609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:45.707973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.319030+0000","last_change":"2024-03-11T19:20:31.325750+0000","last_active":"2024-03-11T19:21:32.319030+0000","last_peered":"2024-03-11T19:21:32.319030+0000","last_clean":"2024-03-11T19:21:32.319030+0000","last_became_active":"2024-03-11T19:20:31.324671+0000","last_became_peered":"2024-03-11T19:20:31.324671+0000","last_unstale":"2024-03-11T19:21:32.319030+0000","last_undegraded":"2024-03-11T19:21:32.319030+0000","last_fullsized":"2024-03-11T19:21:32.319030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:01:17.238573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413060+0000","last_change":"2024-03-11T19:19:42.374433+0000","last_active":"2024-03-11T19:21:32.413060+0000","last_peered":"2024-03-11T19:21:32.413060+0000","last_clean":"2024-03-11T19:21:32.413060+0000","last_became_active":"2024-03-11T19:19:42.371924+0000","last_became_peered":"2024-03-11T19:19:42.371924+0000","last_unstale":"2024-03-11T19:21:32.413060+0000","last_undegraded":"2024-03-11T19:21:32.413060+0000","last_fullsized":"2024-03-11T19:21:32.413060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:37.515381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481181+0000","last_change":"2024-03-11T19:19:42.378885+0000","last_active":"2024-03-11T19:21:32.481181+0000","last_peered":"2024-03-11T19:21:32.481181+0000","last_clean":"2024-03-11T19:21:32.481181+0000","last_became_active":"2024-03-11T19:19:42.378505+0000","last_became_peered":"2024-03-11T19:19:42.378505+0000","last_unstale":"2024-03-11T19:21:32.481181+0000","last_undegraded":"2024-03-11T19:21:32.481181+0000","last_fullsized":"2024-03-11T19:21:32.481181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:12.380431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481048+0000","last_change":"2024-03-11T19:19:42.377500+0000","last_active":"2024-03-11T19:21:32.481048+0000","last_peered":"2024-03-11T19:21:32.481048+0000","last_clean":"2024-03-11T19:21:32.481048+0000","last_became_active":"2024-03-11T19:19:42.377083+0000","last_became_peered":"2024-03-11T19:19:42.377083+0000","last_unstale":"2024-03-11T19:21:32.481048+0000","last_undegraded":"2024-03-11T19:21:32.481048+0000","last_fullsized":"2024-03-11T19:21:32.481048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:02:56.146071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480847+0000","last_change":"2024-03-11T19:19:42.375809+0000","last_active":"2024-03-11T19:21:32.480847+0000","last_peered":"2024-03-11T19:21:32.480847+0000","last_clean":"2024-03-11T19:21:32.480847+0000","last_became_active":"2024-03-11T19:19:42.375422+0000","last_became_peered":"2024-03-11T19:19:42.375422+0000","last_unstale":"2024-03-11T19:21:32.480847+0000","last_undegraded":"2024-03-11T19:21:32.480847+0000","last_fullsized":"2024-03-11T19:21:32.480847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:17:21.344587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481266+0000","last_change":"2024-03-11T19:19:42.381932+0000","last_active":"2024-03-11T19:21:32.481266+0000","last_peered":"2024-03-11T19:21:32.481266+0000","last_clean":"2024-03-11T19:21:32.481266+0000","last_became_active":"2024-03-11T19:19:42.381591+0000","last_became_peered":"2024-03-11T19:19:42.381591+0000","last_unstale":"2024-03-11T19:21:32.481266+0000","last_undegraded":"2024-03-11T19:21:32.481266+0000","last_fullsized":"2024-03-11T19:21:32.481266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:03.619050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318747+0000","last_change":"2024-03-11T19:19:42.374924+0000","last_active":"2024-03-11T19:21:32.318747+0000","last_peered":"2024-03-11T19:21:32.318747+0000","last_clean":"2024-03-11T19:21:32.318747+0000","last_became_active":"2024-03-11T19:19:42.374094+0000","last_became_peered":"2024-03-11T19:19:42.374094+0000","last_unstale":"2024-03-11T19:21:32.318747+0000","last_undegraded":"2024-03-11T19:21:32.318747+0000","last_fullsized":"2024-03-11T19:21:32.318747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:15.327665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313901+0000","last_change":"2024-03-11T19:19:42.381807+0000","last_active":"2024-03-11T19:21:32.313901+0000","last_peered":"2024-03-11T19:21:32.313901+0000","last_clean":"2024-03-11T19:21:32.313901+0000","last_became_active":"2024-03-11T19:19:42.381056+0000","last_became_peered":"2024-03-11T19:19:42.381056+0000","last_unstale":"2024-03-11T19:21:32.313901+0000","last_undegraded":"2024-03-11T19:21:32.313901+0000","last_fullsized":"2024-03-11T19:21:32.313901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:00.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313259+0000","last_change":"2024-03-11T19:19:46.825875+0000","last_active":"2024-03-11T19:21:32.313259+0000","last_peered":"2024-03-11T19:21:32.313259+0000","last_clean":"2024-03-11T19:21:32.313259+0000","last_became_active":"2024-03-11T19:19:46.825528+0000","last_became_peered":"2024-03-11T19:19:46.825528+0000","last_unstale":"2024-03-11T19:21:32.313259+0000","last_undegraded":"2024-03-11T19:21:32.313259+0000","last_fullsized":"2024-03-11T19:21:32.313259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:32:38.667775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318722+0000","last_change":"2024-03-11T19:19:41.393633+0000","last_active":"2024-03-11T19:21:32.318722+0000","last_peered":"2024-03-11T19:21:32.318722+0000","last_clean":"2024-03-11T19:21:32.318722+0000","last_became_active":"2024-03-11T19:19:41.389865+0000","last_became_peered":"2024-03-11T19:19:41.389865+0000","last_unstale":"2024-03-11T19:21:32.318722+0000","last_undegraded":"2024-03-11T19:21:32.318722+0000","last_fullsized":"2024-03-11T19:21:32.318722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:29.251688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315501+0000","last_change":"2024-03-11T19:19:42.392393+0000","last_active":"2024-03-11T19:21:32.315501+0000","last_peered":"2024-03-11T19:21:32.315501+0000","last_clean":"2024-03-11T19:21:32.315501+0000","last_became_active":"2024-03-11T19:19:42.390724+0000","last_became_peered":"2024-03-11T19:19:42.390724+0000","last_unstale":"2024-03-11T19:21:32.315501+0000","last_undegraded":"2024-03-11T19:21:32.315501+0000","last_fullsized":"2024-03-11T19:21:32.315501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:49.410831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416082+0000","last_change":"2024-03-11T19:19:46.817231+0000","last_active":"2024-03-11T19:21:32.416082+0000","last_peered":"2024-03-11T19:21:32.416082+0000","last_clean":"2024-03-11T19:21:32.416082+0000","last_became_active":"2024-03-11T19:19:46.816818+0000","last_became_peered":"2024-03-11T19:19:46.816818+0000","last_unstale":"2024-03-11T19:21:32.416082+0000","last_undegraded":"2024-03-11T19:21:32.416082+0000","last_fullsized":"2024-03-11T19:21:32.416082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:49.562628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315668+0000","last_change":"2024-03-11T19:19:41.401029+0000","last_active":"2024-03-11T19:21:32.315668+0000","last_peered":"2024-03-11T19:21:32.315668+0000","last_clean":"2024-03-11T19:21:32.315668+0000","last_became_active":"2024-03-11T19:19:41.400659+0000","last_became_peered":"2024-03-11T19:19:41.400659+0000","last_unstale":"2024-03-11T19:21:32.315668+0000","last_undegraded":"2024-03-11T19:21:32.315668+0000","last_fullsized":"2024-03-11T19:21:32.315668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:07.903543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481188+0000","last_change":"2024-03-11T19:19:42.380714+0000","last_active":"2024-03-11T19:21:32.481188+0000","last_peered":"2024-03-11T19:21:32.481188+0000","last_clean":"2024-03-11T19:21:32.481188+0000","last_became_active":"2024-03-11T19:19:42.380064+0000","last_became_peered":"2024-03-11T19:19:42.380064+0000","last_unstale":"2024-03-11T19:21:32.481188+0000","last_undegraded":"2024-03-11T19:21:32.481188+0000","last_fullsized":"2024-03-11T19:21:32.481188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:47.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416161+0000","last_change":"2024-03-11T19:19:46.815643+0000","last_active":"2024-03-11T19:21:32.416161+0000","last_peered":"2024-03-11T19:21:32.416161+0000","last_clean":"2024-03-11T19:21:32.416161+0000","last_became_active":"2024-03-11T19:19:46.815222+0000","last_became_peered":"2024-03-11T19:19:46.815222+0000","last_unstale":"2024-03-11T19:21:32.416161+0000","last_undegraded":"2024-03-11T19:21:32.416161+0000","last_fullsized":"2024-03-11T19:21:32.416161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:44:24.625495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322598+0000","last_change":"2024-03-11T19:19:41.397810+0000","last_active":"2024-03-11T19:21:32.322598+0000","last_peered":"2024-03-11T19:21:32.322598+0000","last_clean":"2024-03-11T19:21:32.322598+0000","last_became_active":"2024-03-11T19:19:41.396726+0000","last_became_peered":"2024-03-11T19:19:41.396726+0000","last_unstale":"2024-03-11T19:21:32.322598+0000","last_undegraded":"2024-03-11T19:21:32.322598+0000","last_fullsized":"2024-03-11T19:21:32.322598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:26.328409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481185+0000","last_change":"2024-03-11T19:19:42.360595+0000","last_active":"2024-03-11T19:21:32.481185+0000","last_peered":"2024-03-11T19:21:32.481185+0000","last_clean":"2024-03-11T19:21:32.481185+0000","last_became_active":"2024-03-11T19:19:42.360230+0000","last_became_peered":"2024-03-11T19:19:42.360230+0000","last_unstale":"2024-03-11T19:21:32.481185+0000","last_undegraded":"2024-03-11T19:21:32.481185+0000","last_fullsized":"2024-03-11T19:21:32.481185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:45.480391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416111+0000","last_change":"2024-03-11T19:19:46.817442+0000","last_active":"2024-03-11T19:21:32.416111+0000","last_peered":"2024-03-11T19:21:32.416111+0000","last_clean":"2024-03-11T19:21:32.416111+0000","last_became_active":"2024-03-11T19:19:46.816944+0000","last_became_peered":"2024-03-11T19:19:46.816944+0000","last_unstale":"2024-03-11T19:21:32.416111+0000","last_undegraded":"2024-03-11T19:21:32.416111+0000","last_fullsized":"2024-03-11T19:21:32.416111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:05:43.783450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322442+0000","last_change":"2024-03-11T19:19:41.389608+0000","last_active":"2024-03-11T19:21:32.322442+0000","last_peered":"2024-03-11T19:21:32.322442+0000","last_clean":"2024-03-11T19:21:32.322442+0000","last_became_active":"2024-03-11T19:19:41.389281+0000","last_became_peered":"2024-03-11T19:19:41.389281+0000","last_unstale":"2024-03-11T19:21:32.322442+0000","last_undegraded":"2024-03-11T19:21:32.322442+0000","last_fullsized":"2024-03-11T19:21:32.322442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:46.213494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357364+0000","last_change":"2024-03-11T19:19:42.371444+0000","last_active":"2024-03-11T19:21:32.357364+0000","last_peered":"2024-03-11T19:21:32.357364+0000","last_clean":"2024-03-11T19:21:32.357364+0000","last_became_active":"2024-03-11T19:19:42.371129+0000","last_became_peered":"2024-03-11T19:19:42.371129+0000","last_unstale":"2024-03-11T19:21:32.357364+0000","last_undegraded":"2024-03-11T19:21:32.357364+0000","last_fullsized":"2024-03-11T19:21:32.357364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:04:07.449453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313758+0000","last_change":"2024-03-11T19:19:46.805616+0000","last_active":"2024-03-11T19:21:32.313758+0000","last_peered":"2024-03-11T19:21:32.313758+0000","last_clean":"2024-03-11T19:21:32.313758+0000","last_became_active":"2024-03-11T19:19:46.805069+0000","last_became_peered":"2024-03-11T19:19:46.805069+0000","last_unstale":"2024-03-11T19:21:32.313758+0000","last_undegraded":"2024-03-11T19:21:32.313758+0000","last_fullsized":"2024-03-11T19:21:32.313758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:47:23.197283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:52.385870+0000","last_change":"2024-03-11T19:19:41.392375+0000","last_active":"2024-03-11T19:21:52.385870+0000","last_peered":"2024-03-11T19:21:52.385870+0000","last_clean":"2024-03-11T19:21:52.385870+0000","last_became_active":"2024-03-11T19:19:41.392057+0000","last_became_peered":"2024-03-11T19:19:41.392057+0000","last_unstale":"2024-03-11T19:21:52.385870+0000","last_undegraded":"2024-03-11T19:21:52.385870+0000","last_fullsized":"2024-03-11T19:21:52.385870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:12.842671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322166+0000","last_change":"2024-03-11T19:19:42.373255+0000","last_active":"2024-03-11T19:21:32.322166+0000","last_peered":"2024-03-11T19:21:32.322166+0000","last_clean":"2024-03-11T19:21:32.322166+0000","last_became_active":"2024-03-11T19:19:42.372900+0000","last_became_peered":"2024-03-11T19:19:42.372900+0000","last_unstale":"2024-03-11T19:21:32.322166+0000","last_undegraded":"2024-03-11T19:21:32.322166+0000","last_fullsized":"2024-03-11T19:21:32.322166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:56:45.517189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313365+0000","last_change":"2024-03-11T19:20:31.363743+0000","last_active":"2024-03-11T19:21:32.313365+0000","last_peered":"2024-03-11T19:21:32.313365+0000","last_clean":"2024-03-11T19:21:32.313365+0000","last_became_active":"2024-03-11T19:20:31.363255+0000","last_became_peered":"2024-03-11T19:20:31.363255+0000","last_unstale":"2024-03-11T19:21:32.313365+0000","last_undegraded":"2024-03-11T19:21:32.313365+0000","last_fullsized":"2024-03-11T19:21:32.313365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:51:05.796735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.312970+0000","last_change":"2024-03-11T19:19:41.385591+0000","last_active":"2024-03-11T19:21:32.312970+0000","last_peered":"2024-03-11T19:21:32.312970+0000","last_clean":"2024-03-11T19:21:32.312970+0000","last_became_active":"2024-03-11T19:19:41.385236+0000","last_became_peered":"2024-03-11T19:19:41.385236+0000","last_unstale":"2024-03-11T19:21:32.312970+0000","last_undegraded":"2024-03-11T19:21:32.312970+0000","last_fullsized":"2024-03-11T19:21:32.312970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:55:55.558948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357301+0000","last_change":"2024-03-11T19:19:42.380934+0000","last_active":"2024-03-11T19:21:32.357301+0000","last_peered":"2024-03-11T19:21:32.357301+0000","last_clean":"2024-03-11T19:21:32.357301+0000","last_became_active":"2024-03-11T19:19:42.380602+0000","last_became_peered":"2024-03-11T19:19:42.380602+0000","last_unstale":"2024-03-11T19:21:32.357301+0000","last_undegraded":"2024-03-11T19:21:32.357301+0000","last_fullsized":"2024-03-11T19:21:32.357301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:58:26.813552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481285+0000","last_change":"2024-03-11T19:19:46.814227+0000","last_active":"2024-03-11T19:21:32.481285+0000","last_peered":"2024-03-11T19:21:32.481285+0000","last_clean":"2024-03-11T19:21:32.481285+0000","last_became_active":"2024-03-11T19:19:46.813874+0000","last_became_peered":"2024-03-11T19:19:46.813874+0000","last_unstale":"2024-03-11T19:21:32.481285+0000","last_undegraded":"2024-03-11T19:21:32.481285+0000","last_fullsized":"2024-03-11T19:21:32.481285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:21.493313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412609+0000","last_change":"2024-03-11T19:19:41.392245+0000","last_active":"2024-03-11T19:21:32.412609+0000","last_peered":"2024-03-11T19:21:32.412609+0000","last_clean":"2024-03-11T19:21:32.412609+0000","last_became_active":"2024-03-11T19:19:41.391928+0000","last_became_peered":"2024-03-11T19:19:41.391928+0000","last_unstale":"2024-03-11T19:21:32.412609+0000","last_undegraded":"2024-03-11T19:21:32.412609+0000","last_fullsized":"2024-03-11T19:21:32.412609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:20.556826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314283+0000","last_change":"2024-03-11T19:19:42.389431+0000","last_active":"2024-03-11T19:21:32.314283+0000","last_peered":"2024-03-11T19:21:32.314283+0000","last_clean":"2024-03-11T19:21:32.314283+0000","last_became_active":"2024-03-11T19:19:42.387885+0000","last_became_peered":"2024-03-11T19:19:42.387885+0000","last_unstale":"2024-03-11T19:21:32.314283+0000","last_undegraded":"2024-03-11T19:21:32.314283+0000","last_fullsized":"2024-03-11T19:21:32.314283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:43.753079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481707+0000","last_change":"2024-03-11T19:19:46.823474+0000","last_active":"2024-03-11T19:21:32.481707+0000","last_peered":"2024-03-11T19:21:32.481707+0000","last_clean":"2024-03-11T19:21:32.481707+0000","last_became_active":"2024-03-11T19:19:46.823008+0000","last_became_peered":"2024-03-11T19:19:46.823008+0000","last_unstale":"2024-03-11T19:21:32.481707+0000","last_undegraded":"2024-03-11T19:21:32.481707+0000","last_fullsized":"2024-03-11T19:21:32.481707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:28.381809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356737+0000","last_change":"2024-03-11T19:19:41.395332+0000","last_active":"2024-03-11T19:21:32.356737+0000","last_peered":"2024-03-11T19:21:32.356737+0000","last_clean":"2024-03-11T19:21:32.356737+0000","last_became_active":"2024-03-11T19:19:41.394989+0000","last_became_peered":"2024-03-11T19:19:41.394989+0000","last_unstale":"2024-03-11T19:21:32.356737+0000","last_undegraded":"2024-03-11T19:21:32.356737+0000","last_fullsized":"2024-03-11T19:21:32.356737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:57:31.486891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318863+0000","last_change":"2024-03-11T19:19:42.393223+0000","last_active":"2024-03-11T19:21:32.318863+0000","last_peered":"2024-03-11T19:21:32.318863+0000","last_clean":"2024-03-11T19:21:32.318863+0000","last_became_active":"2024-03-11T19:19:42.392787+0000","last_became_peered":"2024-03-11T19:19:42.392787+0000","last_unstale":"2024-03-11T19:21:32.318863+0000","last_undegraded":"2024-03-11T19:21:32.318863+0000","last_fullsized":"2024-03-11T19:21:32.318863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:32:50.693896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314245+0000","last_change":"2024-03-11T19:19:46.808305+0000","last_active":"2024-03-11T19:21:32.314245+0000","last_peered":"2024-03-11T19:21:32.314245+0000","last_clean":"2024-03-11T19:21:32.314245+0000","last_became_active":"2024-03-11T19:19:46.807816+0000","last_became_peered":"2024-03-11T19:19:46.807816+0000","last_unstale":"2024-03-11T19:21:32.314245+0000","last_undegraded":"2024-03-11T19:21:32.314245+0000","last_fullsized":"2024-03-11T19:21:32.314245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:55:43.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":131,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:50.382324+0000","last_change":"2024-03-11T19:19:41.389335+0000","last_active":"2024-03-11T19:21:50.382324+0000","last_peered":"2024-03-11T19:21:50.382324+0000","last_clean":"2024-03-11T19:21:50.382324+0000","last_became_active":"2024-03-11T19:19:41.389023+0000","last_became_peered":"2024-03-11T19:19:41.389023+0000","last_unstale":"2024-03-11T19:21:50.382324+0000","last_undegraded":"2024-03-11T19:21:50.382324+0000","last_fullsized":"2024-03-11T19:21:50.382324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:12:59.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481552+0000","last_change":"2024-03-11T19:19:42.378379+0000","last_active":"2024-03-11T19:21:32.481552+0000","last_peered":"2024-03-11T19:21:32.481552+0000","last_clean":"2024-03-11T19:21:32.481552+0000","last_became_active":"2024-03-11T19:19:42.378033+0000","last_became_peered":"2024-03-11T19:19:42.378033+0000","last_unstale":"2024-03-11T19:21:32.481552+0000","last_undegraded":"2024-03-11T19:21:32.481552+0000","last_fullsized":"2024-03-11T19:21:32.481552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:24.154420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317132+0000","last_change":"2024-03-11T19:19:46.809464+0000","last_active":"2024-03-11T19:21:32.317132+0000","last_peered":"2024-03-11T19:21:32.317132+0000","last_clean":"2024-03-11T19:21:32.317132+0000","last_became_active":"2024-03-11T19:19:46.808955+0000","last_became_peered":"2024-03-11T19:19:46.808955+0000","last_unstale":"2024-03-11T19:21:32.317132+0000","last_undegraded":"2024-03-11T19:21:32.317132+0000","last_fullsized":"2024-03-11T19:21:32.317132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:42.088992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481875+0000","last_change":"2024-03-11T19:19:41.395619+0000","last_active":"2024-03-11T19:21:32.481875+0000","last_peered":"2024-03-11T19:21:32.481875+0000","last_clean":"2024-03-11T19:21:32.481875+0000","last_became_active":"2024-03-11T19:19:41.394658+0000","last_became_peered":"2024-03-11T19:19:41.394658+0000","last_unstale":"2024-03-11T19:21:32.481875+0000","last_undegraded":"2024-03-11T19:21:32.481875+0000","last_fullsized":"2024-03-11T19:21:32.481875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:10:39.020508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481332+0000","last_change":"2024-03-11T19:19:42.380416+0000","last_active":"2024-03-11T19:21:32.481332+0000","last_peered":"2024-03-11T19:21:32.481332+0000","last_clean":"2024-03-11T19:21:32.481332+0000","last_became_active":"2024-03-11T19:19:42.379625+0000","last_became_peered":"2024-03-11T19:19:42.379625+0000","last_unstale":"2024-03-11T19:21:32.481332+0000","last_undegraded":"2024-03-11T19:21:32.481332+0000","last_fullsized":"2024-03-11T19:21:32.481332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:45:45.374211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416320+0000","last_change":"2024-03-11T19:19:46.808005+0000","last_active":"2024-03-11T19:21:32.416320+0000","last_peered":"2024-03-11T19:21:32.416320+0000","last_clean":"2024-03-11T19:21:32.416320+0000","last_became_active":"2024-03-11T19:19:46.807549+0000","last_became_peered":"2024-03-11T19:19:46.807549+0000","last_unstale":"2024-03-11T19:21:32.416320+0000","last_undegraded":"2024-03-11T19:21:32.416320+0000","last_fullsized":"2024-03-11T19:21:32.416320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:17.171846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357344+0000","last_change":"2024-03-11T19:19:41.398755+0000","last_active":"2024-03-11T19:21:32.357344+0000","last_peered":"2024-03-11T19:21:32.357344+0000","last_clean":"2024-03-11T19:21:32.357344+0000","last_became_active":"2024-03-11T19:19:41.398406+0000","last_became_peered":"2024-03-11T19:19:41.398406+0000","last_unstale":"2024-03-11T19:21:32.357344+0000","last_undegraded":"2024-03-11T19:21:32.357344+0000","last_fullsized":"2024-03-11T19:21:32.357344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:37.793328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481540+0000","last_change":"2024-03-11T19:19:42.364757+0000","last_active":"2024-03-11T19:21:32.481540+0000","last_peered":"2024-03-11T19:21:32.481540+0000","last_clean":"2024-03-11T19:21:32.481540+0000","last_became_active":"2024-03-11T19:19:42.364432+0000","last_became_peered":"2024-03-11T19:19:42.364432+0000","last_unstale":"2024-03-11T19:21:32.481540+0000","last_undegraded":"2024-03-11T19:21:32.481540+0000","last_fullsized":"2024-03-11T19:21:32.481540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:29.226200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317967+0000","last_change":"2024-03-11T19:19:46.809300+0000","last_active":"2024-03-11T19:21:32.317967+0000","last_peered":"2024-03-11T19:21:32.317967+0000","last_clean":"2024-03-11T19:21:32.317967+0000","last_became_active":"2024-03-11T19:19:46.807327+0000","last_became_peered":"2024-03-11T19:19:46.807327+0000","last_unstale":"2024-03-11T19:21:32.317967+0000","last_undegraded":"2024-03-11T19:21:32.317967+0000","last_fullsized":"2024-03-11T19:21:32.317967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:37:54.082436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":271,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:51.383447+0000","last_change":"2024-03-11T19:19:41.392270+0000","last_active":"2024-03-11T19:21:51.383447+0000","last_peered":"2024-03-11T19:21:51.383447+0000","last_clean":"2024-03-11T19:21:51.383447+0000","last_became_active":"2024-03-11T19:19:41.391559+0000","last_became_peered":"2024-03-11T19:19:41.391559+0000","last_unstale":"2024-03-11T19:21:51.383447+0000","last_undegraded":"2024-03-11T19:21:51.383447+0000","last_fullsized":"2024-03-11T19:21:51.383447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:45.655610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":203,"num_read_kb":162,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322449+0000","last_change":"2024-03-11T19:19:42.368323+0000","last_active":"2024-03-11T19:21:32.322449+0000","last_peered":"2024-03-11T19:21:32.322449+0000","last_clean":"2024-03-11T19:21:32.322449+0000","last_became_active":"2024-03-11T19:19:42.367973+0000","last_became_peered":"2024-03-11T19:19:42.367973+0000","last_unstale":"2024-03-11T19:21:32.322449+0000","last_undegraded":"2024-03-11T19:21:32.322449+0000","last_fullsized":"2024-03-11T19:21:32.322449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:11.573348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357646+0000","last_change":"2024-03-11T19:19:46.822255+0000","last_active":"2024-03-11T19:21:32.357646+0000","last_peered":"2024-03-11T19:21:32.357646+0000","last_clean":"2024-03-11T19:21:32.357646+0000","last_became_active":"2024-03-11T19:19:46.821306+0000","last_became_peered":"2024-03-11T19:19:46.821306+0000","last_unstale":"2024-03-11T19:21:32.357646+0000","last_undegraded":"2024-03-11T19:21:32.357646+0000","last_fullsized":"2024-03-11T19:21:32.357646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:22.353676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313431+0000","last_change":"2024-03-11T19:19:41.397723+0000","last_active":"2024-03-11T19:21:32.313431+0000","last_peered":"2024-03-11T19:21:32.313431+0000","last_clean":"2024-03-11T19:21:32.313431+0000","last_became_active":"2024-03-11T19:19:41.397394+0000","last_became_peered":"2024-03-11T19:19:41.397394+0000","last_unstale":"2024-03-11T19:21:32.313431+0000","last_undegraded":"2024-03-11T19:21:32.313431+0000","last_fullsized":"2024-03-11T19:21:32.313431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:48:42.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412447+0000","last_change":"2024-03-11T19:19:42.389254+0000","last_active":"2024-03-11T19:21:32.412447+0000","last_peered":"2024-03-11T19:21:32.412447+0000","last_clean":"2024-03-11T19:21:32.412447+0000","last_became_active":"2024-03-11T19:19:42.388861+0000","last_became_peered":"2024-03-11T19:19:42.388861+0000","last_unstale":"2024-03-11T19:21:32.412447+0000","last_undegraded":"2024-03-11T19:21:32.412447+0000","last_fullsized":"2024-03-11T19:21:32.412447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:03.308802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481035+0000","last_change":"2024-03-11T19:19:46.821667+0000","last_active":"2024-03-11T19:21:32.481035+0000","last_peered":"2024-03-11T19:21:32.481035+0000","last_clean":"2024-03-11T19:21:32.481035+0000","last_became_active":"2024-03-11T19:19:46.821238+0000","last_became_peered":"2024-03-11T19:19:46.821238+0000","last_unstale":"2024-03-11T19:21:32.481035+0000","last_undegraded":"2024-03-11T19:21:32.481035+0000","last_fullsized":"2024-03-11T19:21:32.481035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:30.997248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357102+0000","last_change":"2024-03-11T19:19:41.403277+0000","last_active":"2024-03-11T19:21:32.357102+0000","last_peered":"2024-03-11T19:21:32.357102+0000","last_clean":"2024-03-11T19:21:32.357102+0000","last_became_active":"2024-03-11T19:19:41.402875+0000","last_became_peered":"2024-03-11T19:19:41.402875+0000","last_unstale":"2024-03-11T19:21:32.357102+0000","last_undegraded":"2024-03-11T19:21:32.357102+0000","last_fullsized":"2024-03-11T19:21:32.357102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.937257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481455+0000","last_change":"2024-03-11T19:19:42.376244+0000","last_active":"2024-03-11T19:21:32.481455+0000","last_peered":"2024-03-11T19:21:32.481455+0000","last_clean":"2024-03-11T19:21:32.481455+0000","last_became_active":"2024-03-11T19:19:42.375896+0000","last_became_peered":"2024-03-11T19:19:42.375896+0000","last_unstale":"2024-03-11T19:21:32.481455+0000","last_undegraded":"2024-03-11T19:21:32.481455+0000","last_fullsized":"2024-03-11T19:21:32.481455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:46.295898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317824+0000","last_change":"2024-03-11T19:19:46.811977+0000","last_active":"2024-03-11T19:21:32.317824+0000","last_peered":"2024-03-11T19:21:32.317824+0000","last_clean":"2024-03-11T19:21:32.317824+0000","last_became_active":"2024-03-11T19:19:46.811570+0000","last_became_peered":"2024-03-11T19:19:46.811570+0000","last_unstale":"2024-03-11T19:21:32.317824+0000","last_undegraded":"2024-03-11T19:21:32.317824+0000","last_fullsized":"2024-03-11T19:21:32.317824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:00:18.329040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":134,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:47.452158+0000","last_change":"2024-03-11T19:19:41.387779+0000","last_active":"2024-03-11T19:21:47.452158+0000","last_peered":"2024-03-11T19:21:47.452158+0000","last_clean":"2024-03-11T19:21:47.452158+0000","last_became_active":"2024-03-11T19:19:41.387424+0000","last_became_peered":"2024-03-11T19:19:41.387424+0000","last_unstale":"2024-03-11T19:21:47.452158+0000","last_undegraded":"2024-03-11T19:21:47.452158+0000","last_fullsized":"2024-03-11T19:21:47.452158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:49.936069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480660+0000","last_change":"2024-03-11T19:19:42.367664+0000","last_active":"2024-03-11T19:21:32.480660+0000","last_peered":"2024-03-11T19:21:32.480660+0000","last_clean":"2024-03-11T19:21:32.480660+0000","last_became_active":"2024-03-11T19:19:42.367343+0000","last_became_peered":"2024-03-11T19:19:42.367343+0000","last_unstale":"2024-03-11T19:21:32.480660+0000","last_undegraded":"2024-03-11T19:21:32.480660+0000","last_fullsized":"2024-03-11T19:21:32.480660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:29.215094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414451+0000","last_change":"2024-03-11T19:19:46.821832+0000","last_active":"2024-03-11T19:21:32.414451+0000","last_peered":"2024-03-11T19:21:32.414451+0000","last_clean":"2024-03-11T19:21:32.414451+0000","last_became_active":"2024-03-11T19:19:46.821183+0000","last_became_peered":"2024-03-11T19:19:46.821183+0000","last_unstale":"2024-03-11T19:21:32.414451+0000","last_undegraded":"2024-03-11T19:21:32.414451+0000","last_fullsized":"2024-03-11T19:21:32.414451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:56.293370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322016+0000","last_change":"2024-03-11T19:19:41.400790+0000","last_active":"2024-03-11T19:21:32.322016+0000","last_peered":"2024-03-11T19:21:32.322016+0000","last_clean":"2024-03-11T19:21:32.322016+0000","last_became_active":"2024-03-11T19:19:41.400456+0000","last_became_peered":"2024-03-11T19:19:41.400456+0000","last_unstale":"2024-03-11T19:21:32.322016+0000","last_undegraded":"2024-03-11T19:21:32.322016+0000","last_fullsized":"2024-03-11T19:21:32.322016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:00:18.195113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356909+0000","last_change":"2024-03-11T19:19:42.370608+0000","last_active":"2024-03-11T19:21:32.356909+0000","last_peered":"2024-03-11T19:21:32.356909+0000","last_clean":"2024-03-11T19:21:32.356909+0000","last_became_active":"2024-03-11T19:19:42.370259+0000","last_became_peered":"2024-03-11T19:19:42.370259+0000","last_unstale":"2024-03-11T19:21:32.356909+0000","last_undegraded":"2024-03-11T19:21:32.356909+0000","last_fullsized":"2024-03-11T19:21:32.356909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:50:05.778074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356874+0000","last_change":"2024-03-11T19:19:46.826437+0000","last_active":"2024-03-11T19:21:32.356874+0000","last_peered":"2024-03-11T19:21:32.356874+0000","last_clean":"2024-03-11T19:21:32.356874+0000","last_became_active":"2024-03-11T19:19:46.825964+0000","last_became_peered":"2024-03-11T19:19:46.825964+0000","last_unstale":"2024-03-11T19:21:32.356874+0000","last_undegraded":"2024-03-11T19:21:32.356874+0000","last_fullsized":"2024-03-11T19:21:32.356874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:07.970696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322748+0000","last_change":"2024-03-11T19:19:41.388747+0000","last_active":"2024-03-11T19:21:32.322748+0000","last_peered":"2024-03-11T19:21:32.322748+0000","last_clean":"2024-03-11T19:21:32.322748+0000","last_became_active":"2024-03-11T19:19:41.388408+0000","last_became_peered":"2024-03-11T19:19:41.388408+0000","last_unstale":"2024-03-11T19:21:32.322748+0000","last_undegraded":"2024-03-11T19:21:32.322748+0000","last_fullsized":"2024-03-11T19:21:32.322748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:32:20.195039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481213+0000","last_change":"2024-03-11T19:20:31.328667+0000","last_active":"2024-03-11T19:21:32.481213+0000","last_peered":"2024-03-11T19:21:32.481213+0000","last_clean":"2024-03-11T19:21:32.481213+0000","last_became_active":"2024-03-11T19:20:31.328215+0000","last_became_peered":"2024-03-11T19:20:31.328215+0000","last_unstale":"2024-03-11T19:21:32.481213+0000","last_undegraded":"2024-03-11T19:21:32.481213+0000","last_fullsized":"2024-03-11T19:21:32.481213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:48.188927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322094+0000","last_change":"2024-03-11T19:19:46.934632+0000","last_active":"2024-03-11T19:21:32.322094+0000","last_peered":"2024-03-11T19:21:32.322094+0000","last_clean":"2024-03-11T19:21:32.322094+0000","last_became_active":"2024-03-11T19:19:46.934172+0000","last_became_peered":"2024-03-11T19:19:46.934172+0000","last_unstale":"2024-03-11T19:21:32.322094+0000","last_undegraded":"2024-03-11T19:21:32.322094+0000","last_fullsized":"2024-03-11T19:21:32.322094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:01:21.675154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.319056+0000","last_change":"2024-03-11T19:20:31.340137+0000","last_active":"2024-03-11T19:21:32.319056+0000","last_peered":"2024-03-11T19:21:32.319056+0000","last_clean":"2024-03-11T19:21:32.319056+0000","last_became_active":"2024-03-11T19:20:31.339299+0000","last_became_peered":"2024-03-11T19:20:31.339299+0000","last_unstale":"2024-03-11T19:21:32.319056+0000","last_undegraded":"2024-03-11T19:21:32.319056+0000","last_fullsized":"2024-03-11T19:21:32.319056+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:12:37.489161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313317+0000","last_change":"2024-03-11T19:19:42.387716+0000","last_active":"2024-03-11T19:21:32.313317+0000","last_peered":"2024-03-11T19:21:32.313317+0000","last_clean":"2024-03-11T19:21:32.313317+0000","last_became_active":"2024-03-11T19:19:42.387370+0000","last_became_peered":"2024-03-11T19:19:42.387370+0000","last_unstale":"2024-03-11T19:21:32.313317+0000","last_undegraded":"2024-03-11T19:21:32.313317+0000","last_fullsized":"2024-03-11T19:21:32.313317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:50:22.852284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413164+0000","last_change":"2024-03-11T19:19:46.827980+0000","last_active":"2024-03-11T19:21:32.413164+0000","last_peered":"2024-03-11T19:21:32.413164+0000","last_clean":"2024-03-11T19:21:32.413164+0000","last_became_active":"2024-03-11T19:19:46.827317+0000","last_became_peered":"2024-03-11T19:19:46.827317+0000","last_unstale":"2024-03-11T19:21:32.413164+0000","last_undegraded":"2024-03-11T19:21:32.413164+0000","last_fullsized":"2024-03-11T19:21:32.413164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:55:15.266791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481246+0000","last_change":"2024-03-11T19:19:41.394714+0000","last_active":"2024-03-11T19:21:32.481246+0000","last_peered":"2024-03-11T19:21:32.481246+0000","last_clean":"2024-03-11T19:21:32.481246+0000","last_became_active":"2024-03-11T19:19:41.394404+0000","last_became_peered":"2024-03-11T19:19:41.394404+0000","last_unstale":"2024-03-11T19:21:32.481246+0000","last_undegraded":"2024-03-11T19:21:32.481246+0000","last_fullsized":"2024-03-11T19:21:32.481246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:15.960037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313130+0000","last_change":"2024-03-11T19:19:42.381062+0000","last_active":"2024-03-11T19:21:32.313130+0000","last_peered":"2024-03-11T19:21:32.313130+0000","last_clean":"2024-03-11T19:21:32.313130+0000","last_became_active":"2024-03-11T19:19:42.380744+0000","last_became_peered":"2024-03-11T19:19:42.380744+0000","last_unstale":"2024-03-11T19:21:32.313130+0000","last_undegraded":"2024-03-11T19:21:32.313130+0000","last_fullsized":"2024-03-11T19:21:32.313130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:08.767877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414564+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:21:32.414564+0000","last_peered":"2024-03-11T19:21:32.414564+0000","last_clean":"2024-03-11T19:21:32.414564+0000","last_became_active":"2024-03-11T19:19:46.934195+0000","last_became_peered":"2024-03-11T19:19:46.934195+0000","last_unstale":"2024-03-11T19:21:32.414564+0000","last_undegraded":"2024-03-11T19:21:32.414564+0000","last_fullsized":"2024-03-11T19:21:32.414564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:33:19.268162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356902+0000","last_change":"2024-03-11T19:19:42.366940+0000","last_active":"2024-03-11T19:21:32.356902+0000","last_peered":"2024-03-11T19:21:32.356902+0000","last_clean":"2024-03-11T19:21:32.356902+0000","last_became_active":"2024-03-11T19:19:42.366628+0000","last_became_peered":"2024-03-11T19:19:42.366628+0000","last_unstale":"2024-03-11T19:21:32.356902+0000","last_undegraded":"2024-03-11T19:21:32.356902+0000","last_fullsized":"2024-03-11T19:21:32.356902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:05:37.175276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":133,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:49.456069+0000","last_change":"2024-03-11T19:19:41.389919+0000","last_active":"2024-03-11T19:21:49.456069+0000","last_peered":"2024-03-11T19:21:49.456069+0000","last_clean":"2024-03-11T19:21:49.456069+0000","last_became_active":"2024-03-11T19:19:41.389471+0000","last_became_peered":"2024-03-11T19:19:41.389471+0000","last_unstale":"2024-03-11T19:21:49.456069+0000","last_undegraded":"2024-03-11T19:21:49.456069+0000","last_fullsized":"2024-03-11T19:21:49.456069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:41.068642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481111+0000","last_change":"2024-03-11T19:20:31.339329+0000","last_active":"2024-03-11T19:21:32.481111+0000","last_peered":"2024-03-11T19:21:32.481111+0000","last_clean":"2024-03-11T19:21:32.481111+0000","last_became_active":"2024-03-11T19:20:31.328906+0000","last_became_peered":"2024-03-11T19:20:31.328906+0000","last_unstale":"2024-03-11T19:21:32.481111+0000","last_undegraded":"2024-03-11T19:21:32.481111+0000","last_fullsized":"2024-03-11T19:21:32.481111+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:31:46.707878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314799+0000","last_change":"2024-03-11T19:20:31.364729+0000","last_active":"2024-03-11T19:21:32.314799+0000","last_peered":"2024-03-11T19:21:32.314799+0000","last_clean":"2024-03-11T19:21:32.314799+0000","last_became_active":"2024-03-11T19:20:31.363713+0000","last_became_peered":"2024-03-11T19:20:31.363713+0000","last_unstale":"2024-03-11T19:21:32.314799+0000","last_undegraded":"2024-03-11T19:21:32.314799+0000","last_fullsized":"2024-03-11T19:21:32.314799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:53.829080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416389+0000","last_change":"2024-03-11T19:19:41.391501+0000","last_active":"2024-03-11T19:21:32.416389+0000","last_peered":"2024-03-11T19:21:32.416389+0000","last_clean":"2024-03-11T19:21:32.416389+0000","last_became_active":"2024-03-11T19:19:41.391148+0000","last_became_peered":"2024-03-11T19:19:41.391148+0000","last_unstale":"2024-03-11T19:21:32.416389+0000","last_undegraded":"2024-03-11T19:21:32.416389+0000","last_fullsized":"2024-03-11T19:21:32.416389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:46:18.957547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480521+0000","last_change":"2024-03-11T19:18:21.278121+0000","last_active":"2024-03-11T19:21:32.480521+0000","last_peered":"2024-03-11T19:21:32.480521+0000","last_clean":"2024-03-11T19:21:32.480521+0000","last_became_active":"2024-03-11T19:18:21.277678+0000","last_became_peered":"2024-03-11T19:18:21.277678+0000","last_unstale":"2024-03-11T19:21:32.480521+0000","last_undegraded":"2024-03-11T19:21:32.480521+0000","last_fullsized":"2024-03-11T19:21:32.480521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_clean_scrub_stamp":"2024-03-11T19:14:20.582658+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:19.162992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357426+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:21:32.357426+0000","last_peered":"2024-03-11T19:21:32.357426+0000","last_clean":"2024-03-11T19:21:32.357426+0000","last_became_active":"2024-03-11T19:19:46.934304+0000","last_became_peered":"2024-03-11T19:19:46.934304+0000","last_unstale":"2024-03-11T19:21:32.357426+0000","last_undegraded":"2024-03-11T19:21:32.357426+0000","last_fullsized":"2024-03-11T19:21:32.357426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:34:47.392139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412388+0000","last_change":"2024-03-11T19:19:42.363283+0000","last_active":"2024-03-11T19:21:32.412388+0000","last_peered":"2024-03-11T19:21:32.412388+0000","last_clean":"2024-03-11T19:21:32.412388+0000","last_became_active":"2024-03-11T19:19:42.362930+0000","last_became_peered":"2024-03-11T19:19:42.362930+0000","last_unstale":"2024-03-11T19:21:32.412388+0000","last_undegraded":"2024-03-11T19:21:32.412388+0000","last_fullsized":"2024-03-11T19:21:32.412388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:16:36.141585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480649+0000","last_change":"2024-03-11T19:19:41.391776+0000","last_active":"2024-03-11T19:21:32.480649+0000","last_peered":"2024-03-11T19:21:32.480649+0000","last_clean":"2024-03-11T19:21:32.480649+0000","last_became_active":"2024-03-11T19:19:41.391459+0000","last_became_peered":"2024-03-11T19:19:41.391459+0000","last_unstale":"2024-03-11T19:21:32.480649+0000","last_undegraded":"2024-03-11T19:21:32.480649+0000","last_fullsized":"2024-03-11T19:21:32.480649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:36.457307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480919+0000","last_change":"2024-03-11T19:19:46.821662+0000","last_active":"2024-03-11T19:21:32.480919+0000","last_peered":"2024-03-11T19:21:32.480919+0000","last_clean":"2024-03-11T19:21:32.480919+0000","last_became_active":"2024-03-11T19:19:46.821199+0000","last_became_peered":"2024-03-11T19:19:46.821199+0000","last_unstale":"2024-03-11T19:21:32.480919+0000","last_undegraded":"2024-03-11T19:21:32.480919+0000","last_fullsized":"2024-03-11T19:21:32.480919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:08:39.658398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480994+0000","last_change":"2024-03-11T19:20:31.344359+0000","last_active":"2024-03-11T19:21:32.480994+0000","last_peered":"2024-03-11T19:21:32.480994+0000","last_clean":"2024-03-11T19:21:32.480994+0000","last_became_active":"2024-03-11T19:20:31.339153+0000","last_became_peered":"2024-03-11T19:20:31.339153+0000","last_unstale":"2024-03-11T19:21:32.480994+0000","last_undegraded":"2024-03-11T19:21:32.480994+0000","last_fullsized":"2024-03-11T19:21:32.480994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:18:35.854043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413729+0000","last_change":"2024-03-11T19:19:42.380304+0000","last_active":"2024-03-11T19:21:32.413729+0000","last_peered":"2024-03-11T19:21:32.413729+0000","last_clean":"2024-03-11T19:21:32.413729+0000","last_became_active":"2024-03-11T19:19:42.378768+0000","last_became_peered":"2024-03-11T19:19:42.378768+0000","last_unstale":"2024-03-11T19:21:32.413729+0000","last_undegraded":"2024-03-11T19:21:32.413729+0000","last_fullsized":"2024-03-11T19:21:32.413729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:31.465876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314259+0000","last_change":"2024-03-11T19:19:46.934262+0000","last_active":"2024-03-11T19:21:32.314259+0000","last_peered":"2024-03-11T19:21:32.314259+0000","last_clean":"2024-03-11T19:21:32.314259+0000","last_became_active":"2024-03-11T19:19:46.933557+0000","last_became_peered":"2024-03-11T19:19:46.933557+0000","last_unstale":"2024-03-11T19:21:32.314259+0000","last_undegraded":"2024-03-11T19:21:32.314259+0000","last_fullsized":"2024-03-11T19:21:32.314259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:29.227551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318539+0000","last_change":"2024-03-11T19:19:41.402974+0000","last_active":"2024-03-11T19:21:32.318539+0000","last_peered":"2024-03-11T19:21:32.318539+0000","last_clean":"2024-03-11T19:21:32.318539+0000","last_became_active":"2024-03-11T19:19:41.402257+0000","last_became_peered":"2024-03-11T19:19:41.402257+0000","last_unstale":"2024-03-11T19:21:32.318539+0000","last_undegraded":"2024-03-11T19:21:32.318539+0000","last_fullsized":"2024-03-11T19:21:32.318539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:00:27.136592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315380+0000","last_change":"2024-03-11T19:19:42.385331+0000","last_active":"2024-03-11T19:21:32.315380+0000","last_peered":"2024-03-11T19:21:32.315380+0000","last_clean":"2024-03-11T19:21:32.315380+0000","last_became_active":"2024-03-11T19:19:42.381750+0000","last_became_peered":"2024-03-11T19:19:42.381750+0000","last_unstale":"2024-03-11T19:21:32.315380+0000","last_undegraded":"2024-03-11T19:21:32.315380+0000","last_fullsized":"2024-03-11T19:21:32.315380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:54:20.851516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313057+0000","last_change":"2024-03-11T19:19:46.801276+0000","last_active":"2024-03-11T19:21:32.313057+0000","last_peered":"2024-03-11T19:21:32.313057+0000","last_clean":"2024-03-11T19:21:32.313057+0000","last_became_active":"2024-03-11T19:19:46.800791+0000","last_became_peered":"2024-03-11T19:19:46.800791+0000","last_unstale":"2024-03-11T19:21:32.313057+0000","last_undegraded":"2024-03-11T19:21:32.313057+0000","last_fullsized":"2024-03-11T19:21:32.313057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:05.418175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416620+0000","last_change":"2024-03-11T19:19:41.392788+0000","last_active":"2024-03-11T19:21:32.416620+0000","last_peered":"2024-03-11T19:21:32.416620+0000","last_clean":"2024-03-11T19:21:32.416620+0000","last_became_active":"2024-03-11T19:19:41.392483+0000","last_became_peered":"2024-03-11T19:19:41.392483+0000","last_unstale":"2024-03-11T19:21:32.416620+0000","last_undegraded":"2024-03-11T19:21:32.416620+0000","last_fullsized":"2024-03-11T19:21:32.416620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:55.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481752+0000","last_change":"2024-03-11T19:19:42.394123+0000","last_active":"2024-03-11T19:21:32.481752+0000","last_peered":"2024-03-11T19:21:32.481752+0000","last_clean":"2024-03-11T19:21:32.481752+0000","last_became_active":"2024-03-11T19:19:42.393785+0000","last_became_peered":"2024-03-11T19:19:42.393785+0000","last_unstale":"2024-03-11T19:21:32.481752+0000","last_undegraded":"2024-03-11T19:21:32.481752+0000","last_fullsized":"2024-03-11T19:21:32.481752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:46:24.299553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416697+0000","last_change":"2024-03-11T19:20:31.328899+0000","last_active":"2024-03-11T19:21:32.416697+0000","last_peered":"2024-03-11T19:21:32.416697+0000","last_clean":"2024-03-11T19:21:32.416697+0000","last_became_active":"2024-03-11T19:20:31.328401+0000","last_became_peered":"2024-03-11T19:20:31.328401+0000","last_unstale":"2024-03-11T19:21:32.416697+0000","last_undegraded":"2024-03-11T19:21:32.416697+0000","last_fullsized":"2024-03-11T19:21:32.416697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:41.749529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480837+0000","last_change":"2024-03-11T19:19:41.394099+0000","last_active":"2024-03-11T19:21:32.480837+0000","last_peered":"2024-03-11T19:21:32.480837+0000","last_clean":"2024-03-11T19:21:32.480837+0000","last_became_active":"2024-03-11T19:19:41.393780+0000","last_became_peered":"2024-03-11T19:19:41.393780+0000","last_unstale":"2024-03-11T19:21:32.480837+0000","last_undegraded":"2024-03-11T19:21:32.480837+0000","last_fullsized":"2024-03-11T19:21:32.480837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:38:16.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322020+0000","last_change":"2024-03-11T19:19:42.363963+0000","last_active":"2024-03-11T19:21:32.322020+0000","last_peered":"2024-03-11T19:21:32.322020+0000","last_clean":"2024-03-11T19:21:32.322020+0000","last_became_active":"2024-03-11T19:19:42.363315+0000","last_became_peered":"2024-03-11T19:19:42.363315+0000","last_unstale":"2024-03-11T19:21:32.322020+0000","last_undegraded":"2024-03-11T19:21:32.322020+0000","last_fullsized":"2024-03-11T19:21:32.322020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:13:14.828230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318201+0000","last_change":"2024-03-11T19:19:46.808879+0000","last_active":"2024-03-11T19:21:32.318201+0000","last_peered":"2024-03-11T19:21:32.318201+0000","last_clean":"2024-03-11T19:21:32.318201+0000","last_became_active":"2024-03-11T19:19:46.808428+0000","last_became_peered":"2024-03-11T19:19:46.808428+0000","last_unstale":"2024-03-11T19:21:32.318201+0000","last_undegraded":"2024-03-11T19:21:32.318201+0000","last_fullsized":"2024-03-11T19:21:32.318201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:19.024854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314925+0000","last_change":"2024-03-11T19:19:41.395487+0000","last_active":"2024-03-11T19:21:32.314925+0000","last_peered":"2024-03-11T19:21:32.314925+0000","last_clean":"2024-03-11T19:21:32.314925+0000","last_became_active":"2024-03-11T19:19:41.395081+0000","last_became_peered":"2024-03-11T19:19:41.395081+0000","last_unstale":"2024-03-11T19:21:32.314925+0000","last_undegraded":"2024-03-11T19:21:32.314925+0000","last_fullsized":"2024-03-11T19:21:32.314925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:54:58.855745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480941+0000","last_change":"2024-03-11T19:19:42.378860+0000","last_active":"2024-03-11T19:21:32.480941+0000","last_peered":"2024-03-11T19:21:32.480941+0000","last_clean":"2024-03-11T19:21:32.480941+0000","last_became_active":"2024-03-11T19:19:42.378421+0000","last_became_peered":"2024-03-11T19:19:42.378421+0000","last_unstale":"2024-03-11T19:21:32.480941+0000","last_undegraded":"2024-03-11T19:21:32.480941+0000","last_fullsized":"2024-03-11T19:21:32.480941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:08.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416822+0000","last_change":"2024-03-11T19:19:46.815770+0000","last_active":"2024-03-11T19:21:32.416822+0000","last_peered":"2024-03-11T19:21:32.416822+0000","last_clean":"2024-03-11T19:21:32.416822+0000","last_became_active":"2024-03-11T19:19:46.814982+0000","last_became_peered":"2024-03-11T19:19:46.814982+0000","last_unstale":"2024-03-11T19:21:32.416822+0000","last_undegraded":"2024-03-11T19:21:32.416822+0000","last_fullsized":"2024-03-11T19:21:32.416822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:19.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313191+0000","last_change":"2024-03-11T19:19:41.397753+0000","last_active":"2024-03-11T19:21:32.313191+0000","last_peered":"2024-03-11T19:21:32.313191+0000","last_clean":"2024-03-11T19:21:32.313191+0000","last_became_active":"2024-03-11T19:19:41.397200+0000","last_became_peered":"2024-03-11T19:19:41.397200+0000","last_unstale":"2024-03-11T19:21:32.313191+0000","last_undegraded":"2024-03-11T19:21:32.313191+0000","last_fullsized":"2024-03-11T19:21:32.313191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:38:38.326990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314742+0000","last_change":"2024-03-11T19:19:42.387285+0000","last_active":"2024-03-11T19:21:32.314742+0000","last_peered":"2024-03-11T19:21:32.314742+0000","last_clean":"2024-03-11T19:21:32.314742+0000","last_became_active":"2024-03-11T19:19:42.386919+0000","last_became_peered":"2024-03-11T19:19:42.386919+0000","last_unstale":"2024-03-11T19:21:32.314742+0000","last_undegraded":"2024-03-11T19:21:32.314742+0000","last_fullsized":"2024-03-11T19:21:32.314742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.773177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416483+0000","last_change":"2024-03-11T19:19:46.934177+0000","last_active":"2024-03-11T19:21:32.416483+0000","last_peered":"2024-03-11T19:21:32.416483+0000","last_clean":"2024-03-11T19:21:32.416483+0000","last_became_active":"2024-03-11T19:19:46.933699+0000","last_became_peered":"2024-03-11T19:19:46.933699+0000","last_unstale":"2024-03-11T19:21:32.416483+0000","last_undegraded":"2024-03-11T19:21:32.416483+0000","last_fullsized":"2024-03-11T19:21:32.416483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:09:17.433172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322216+0000","last_change":"2024-03-11T19:19:41.396888+0000","last_active":"2024-03-11T19:21:32.322216+0000","last_peered":"2024-03-11T19:21:32.322216+0000","last_clean":"2024-03-11T19:21:32.322216+0000","last_became_active":"2024-03-11T19:19:41.396640+0000","last_became_peered":"2024-03-11T19:19:41.396640+0000","last_unstale":"2024-03-11T19:21:32.322216+0000","last_undegraded":"2024-03-11T19:21:32.322216+0000","last_fullsized":"2024-03-11T19:21:32.322216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:00.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481382+0000","last_change":"2024-03-11T19:19:42.370020+0000","last_active":"2024-03-11T19:21:32.481382+0000","last_peered":"2024-03-11T19:21:32.481382+0000","last_clean":"2024-03-11T19:21:32.481382+0000","last_became_active":"2024-03-11T19:19:42.369355+0000","last_became_peered":"2024-03-11T19:19:42.369355+0000","last_unstale":"2024-03-11T19:21:32.481382+0000","last_undegraded":"2024-03-11T19:21:32.481382+0000","last_fullsized":"2024-03-11T19:21:32.481382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:07:13.572419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480750+0000","last_change":"2024-03-11T19:19:46.818424+0000","last_active":"2024-03-11T19:21:32.480750+0000","last_peered":"2024-03-11T19:21:32.480750+0000","last_clean":"2024-03-11T19:21:32.480750+0000","last_became_active":"2024-03-11T19:19:46.817934+0000","last_became_peered":"2024-03-11T19:19:46.817934+0000","last_unstale":"2024-03-11T19:21:32.480750+0000","last_undegraded":"2024-03-11T19:21:32.480750+0000","last_fullsized":"2024-03-11T19:21:32.480750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:52:56.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313627+0000","last_change":"2024-03-11T19:19:41.399466+0000","last_active":"2024-03-11T19:21:32.313627+0000","last_peered":"2024-03-11T19:21:32.313627+0000","last_clean":"2024-03-11T19:21:32.313627+0000","last_became_active":"2024-03-11T19:19:41.399118+0000","last_became_peered":"2024-03-11T19:19:41.399118+0000","last_unstale":"2024-03-11T19:21:32.313627+0000","last_undegraded":"2024-03-11T19:21:32.313627+0000","last_fullsized":"2024-03-11T19:21:32.313627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:12:46.100596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481727+0000","last_change":"2024-03-11T19:19:42.365437+0000","last_active":"2024-03-11T19:21:32.481727+0000","last_peered":"2024-03-11T19:21:32.481727+0000","last_clean":"2024-03-11T19:21:32.481727+0000","last_became_active":"2024-03-11T19:19:42.365084+0000","last_became_peered":"2024-03-11T19:19:42.365084+0000","last_unstale":"2024-03-11T19:21:32.481727+0000","last_undegraded":"2024-03-11T19:21:32.481727+0000","last_fullsized":"2024-03-11T19:21:32.481727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:35.239100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481460+0000","last_change":"2024-03-11T19:19:46.825067+0000","last_active":"2024-03-11T19:21:32.481460+0000","last_peered":"2024-03-11T19:21:32.481460+0000","last_clean":"2024-03-11T19:21:32.481460+0000","last_became_active":"2024-03-11T19:19:46.821438+0000","last_became_peered":"2024-03-11T19:19:46.821438+0000","last_unstale":"2024-03-11T19:21:32.481460+0000","last_undegraded":"2024-03-11T19:21:32.481460+0000","last_fullsized":"2024-03-11T19:21:32.481460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:08.828669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357475+0000","last_change":"2024-03-11T19:19:41.403788+0000","last_active":"2024-03-11T19:21:32.357475+0000","last_peered":"2024-03-11T19:21:32.357475+0000","last_clean":"2024-03-11T19:21:32.357475+0000","last_became_active":"2024-03-11T19:19:41.403479+0000","last_became_peered":"2024-03-11T19:19:41.403479+0000","last_unstale":"2024-03-11T19:21:32.357475+0000","last_undegraded":"2024-03-11T19:21:32.357475+0000","last_fullsized":"2024-03-11T19:21:32.357475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:58:03.407100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480967+0000","last_change":"2024-03-11T19:19:42.381891+0000","last_active":"2024-03-11T19:21:32.480967+0000","last_peered":"2024-03-11T19:21:32.480967+0000","last_clean":"2024-03-11T19:21:32.480967+0000","last_became_active":"2024-03-11T19:19:42.381573+0000","last_became_peered":"2024-03-11T19:19:42.381573+0000","last_unstale":"2024-03-11T19:21:32.480967+0000","last_undegraded":"2024-03-11T19:21:32.480967+0000","last_fullsized":"2024-03-11T19:21:32.480967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:25.967035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.321995+0000","last_change":"2024-03-11T19:20:31.361853+0000","last_active":"2024-03-11T19:21:32.321995+0000","last_peered":"2024-03-11T19:21:32.321995+0000","last_clean":"2024-03-11T19:21:32.321995+0000","last_became_active":"2024-03-11T19:20:31.361381+0000","last_became_peered":"2024-03-11T19:20:31.361381+0000","last_unstale":"2024-03-11T19:21:32.321995+0000","last_undegraded":"2024-03-11T19:21:32.321995+0000","last_fullsized":"2024-03-11T19:21:32.321995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:56:49.225542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414640+0000","last_change":"2024-03-11T19:19:41.394613+0000","last_active":"2024-03-11T19:21:32.414640+0000","last_peered":"2024-03-11T19:21:32.414640+0000","last_clean":"2024-03-11T19:21:32.414640+0000","last_became_active":"2024-03-11T19:19:41.394252+0000","last_became_peered":"2024-03-11T19:19:41.394252+0000","last_unstale":"2024-03-11T19:21:32.414640+0000","last_undegraded":"2024-03-11T19:21:32.414640+0000","last_fullsized":"2024-03-11T19:21:32.414640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:05:50.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.482039+0000","last_change":"2024-03-11T19:19:42.366706+0000","last_active":"2024-03-11T19:21:32.482039+0000","last_peered":"2024-03-11T19:21:32.482039+0000","last_clean":"2024-03-11T19:21:32.482039+0000","last_became_active":"2024-03-11T19:19:42.366389+0000","last_became_peered":"2024-03-11T19:19:42.366389+0000","last_unstale":"2024-03-11T19:21:32.482039+0000","last_undegraded":"2024-03-11T19:21:32.482039+0000","last_fullsized":"2024-03-11T19:21:32.482039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:15:08.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481070+0000","last_change":"2024-03-11T19:19:42.367886+0000","last_active":"2024-03-11T19:21:32.481070+0000","last_peered":"2024-03-11T19:21:32.481070+0000","last_clean":"2024-03-11T19:21:32.481070+0000","last_became_active":"2024-03-11T19:19:42.367543+0000","last_became_peered":"2024-03-11T19:19:42.367543+0000","last_unstale":"2024-03-11T19:21:32.481070+0000","last_undegraded":"2024-03-11T19:21:32.481070+0000","last_fullsized":"2024-03-11T19:21:32.481070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:15:01.606205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313549+0000","last_change":"2024-03-11T19:19:42.370189+0000","last_active":"2024-03-11T19:21:32.313549+0000","last_peered":"2024-03-11T19:21:32.313549+0000","last_clean":"2024-03-11T19:21:32.313549+0000","last_became_active":"2024-03-11T19:19:42.369553+0000","last_became_peered":"2024-03-11T19:19:42.369553+0000","last_unstale":"2024-03-11T19:21:32.313549+0000","last_undegraded":"2024-03-11T19:21:32.313549+0000","last_fullsized":"2024-03-11T19:21:32.313549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:10:09.973215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313241+0000","last_change":"2024-03-11T19:19:42.380853+0000","last_active":"2024-03-11T19:21:32.313241+0000","last_peered":"2024-03-11T19:21:32.313241+0000","last_clean":"2024-03-11T19:21:32.313241+0000","last_became_active":"2024-03-11T19:19:42.380502+0000","last_became_peered":"2024-03-11T19:19:42.380502+0000","last_unstale":"2024-03-11T19:21:32.313241+0000","last_undegraded":"2024-03-11T19:21:32.313241+0000","last_fullsized":"2024-03-11T19:21:32.313241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:22:20.832801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481764+0000","last_change":"2024-03-11T19:19:42.382850+0000","last_active":"2024-03-11T19:21:32.481764+0000","last_peered":"2024-03-11T19:21:32.481764+0000","last_clean":"2024-03-11T19:21:32.481764+0000","last_became_active":"2024-03-11T19:19:42.382134+0000","last_became_peered":"2024-03-11T19:19:42.382134+0000","last_unstale":"2024-03-11T19:21:32.481764+0000","last_undegraded":"2024-03-11T19:21:32.481764+0000","last_fullsized":"2024-03-11T19:21:32.481764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:49.859290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413599+0000","last_change":"2024-03-11T19:19:42.386882+0000","last_active":"2024-03-11T19:21:32.413599+0000","last_peered":"2024-03-11T19:21:32.413599+0000","last_clean":"2024-03-11T19:21:32.413599+0000","last_became_active":"2024-03-11T19:19:42.386280+0000","last_became_peered":"2024-03-11T19:19:42.386280+0000","last_unstale":"2024-03-11T19:21:32.413599+0000","last_undegraded":"2024-03-11T19:21:32.413599+0000","last_fullsized":"2024-03-11T19:21:32.413599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:30:10.891093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481338+0000","last_change":"2024-03-11T19:19:42.368025+0000","last_active":"2024-03-11T19:21:32.481338+0000","last_peered":"2024-03-11T19:21:32.481338+0000","last_clean":"2024-03-11T19:21:32.481338+0000","last_became_active":"2024-03-11T19:19:42.367695+0000","last_became_peered":"2024-03-11T19:19:42.367695+0000","last_unstale":"2024-03-11T19:21:32.481338+0000","last_undegraded":"2024-03-11T19:21:32.481338+0000","last_fullsized":"2024-03-11T19:21:32.481338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:11:44.776358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313933+0000","last_change":"2024-03-11T19:19:42.364940+0000","last_active":"2024-03-11T19:21:32.313933+0000","last_peered":"2024-03-11T19:21:32.313933+0000","last_clean":"2024-03-11T19:21:32.313933+0000","last_became_active":"2024-03-11T19:19:42.364620+0000","last_became_peered":"2024-03-11T19:19:42.364620+0000","last_unstale":"2024-03-11T19:21:32.313933+0000","last_undegraded":"2024-03-11T19:21:32.313933+0000","last_fullsized":"2024-03-11T19:21:32.313933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:11.415716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315591+0000","last_change":"2024-03-11T19:19:42.392468+0000","last_active":"2024-03-11T19:21:32.315591+0000","last_peered":"2024-03-11T19:21:32.315591+0000","last_clean":"2024-03-11T19:21:32.315591+0000","last_became_active":"2024-03-11T19:19:42.390521+0000","last_became_peered":"2024-03-11T19:19:42.390521+0000","last_unstale":"2024-03-11T19:21:32.315591+0000","last_undegraded":"2024-03-11T19:21:32.315591+0000","last_fullsized":"2024-03-11T19:21:32.315591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:51:21.487796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356884+0000","last_change":"2024-03-11T19:19:42.393130+0000","last_active":"2024-03-11T19:21:32.356884+0000","last_peered":"2024-03-11T19:21:32.356884+0000","last_clean":"2024-03-11T19:21:32.356884+0000","last_became_active":"2024-03-11T19:19:42.392845+0000","last_became_peered":"2024-03-11T19:19:42.392845+0000","last_unstale":"2024-03-11T19:21:32.356884+0000","last_undegraded":"2024-03-11T19:21:32.356884+0000","last_fullsized":"2024-03-11T19:21:32.356884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:17:27.278574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315097+0000","last_change":"2024-03-11T19:19:42.385890+0000","last_active":"2024-03-11T19:21:32.315097+0000","last_peered":"2024-03-11T19:21:32.315097+0000","last_clean":"2024-03-11T19:21:32.315097+0000","last_became_active":"2024-03-11T19:19:42.381939+0000","last_became_peered":"2024-03-11T19:19:42.381939+0000","last_unstale":"2024-03-11T19:21:32.315097+0000","last_undegraded":"2024-03-11T19:21:32.315097+0000","last_fullsized":"2024-03-11T19:21:32.315097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:47:11.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481168+0000","last_change":"2024-03-11T19:19:42.392856+0000","last_active":"2024-03-11T19:21:32.481168+0000","last_peered":"2024-03-11T19:21:32.481168+0000","last_clean":"2024-03-11T19:21:32.481168+0000","last_became_active":"2024-03-11T19:19:42.392554+0000","last_became_peered":"2024-03-11T19:19:42.392554+0000","last_unstale":"2024-03-11T19:21:32.481168+0000","last_undegraded":"2024-03-11T19:21:32.481168+0000","last_fullsized":"2024-03-11T19:21:32.481168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:54:04.840145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412859+0000","last_change":"2024-03-11T19:19:42.383082+0000","last_active":"2024-03-11T19:21:32.412859+0000","last_peered":"2024-03-11T19:21:32.412859+0000","last_clean":"2024-03-11T19:21:32.412859+0000","last_became_active":"2024-03-11T19:19:42.381546+0000","last_became_peered":"2024-03-11T19:19:42.381546+0000","last_unstale":"2024-03-11T19:21:32.412859+0000","last_undegraded":"2024-03-11T19:21:32.412859+0000","last_fullsized":"2024-03-11T19:21:32.412859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:20.493148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481038+0000","last_change":"2024-03-11T19:19:42.369136+0000","last_active":"2024-03-11T19:21:32.481038+0000","last_peered":"2024-03-11T19:21:32.481038+0000","last_clean":"2024-03-11T19:21:32.481038+0000","last_became_active":"2024-03-11T19:19:42.368807+0000","last_became_peered":"2024-03-11T19:19:42.368807+0000","last_unstale":"2024-03-11T19:21:32.481038+0000","last_undegraded":"2024-03-11T19:21:32.481038+0000","last_fullsized":"2024-03-11T19:21:32.481038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:16:34.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281180,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66724,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":608,"num_read_kb":374,"num_write":170,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":200172,"data_compressed":4818,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":523154,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":1177610,"data_compressed":64022,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":523368,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":523266,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1119338,"data_compressed":62416,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":518544,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":581036,"data_compressed":28258,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":519932,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":582386,"data_compressed":28258,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":518738,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":523726,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1114394,"data_compressed":62416,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62582,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63932,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64692,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:21:56.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:21:56.844 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:21:57.354 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:57 smithi072 ceph-mon[31695]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:21:57.355 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:57 smithi072 ceph-mon[31695]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:57 smithi106 ceph-mon[35889]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:21:57.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:57 smithi106 ceph-mon[35889]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:57 smithi143 ceph-mon[36013]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:21:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:57 smithi143 ceph-mon[36013]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:21:59.532 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:21:59 smithi072 ceph-mon[31695]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 -- 172.21.15.72:0/2186648444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 msgr2=0x7f5d54108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/2186648444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 0x7f5d54108080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d440099c0 tx=0x7f5d44009cd0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 -- 172.21.15.72:0/2186648444 shutdown_connections 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/2186648444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d5410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/2186648444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 0x7f5d54108080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/2186648444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 -- 172.21.15.72:0/2186648444 >> 172.21.15.72:0/2186648444 conn(0x7f5d54078da0 msgr2=0x7f5d540791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 -- 172.21.15.72:0/2186648444 shutdown_connections 2024-03-11T19:21:59.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.528+0000 7f5d5aa00700 1 -- 172.21.15.72:0/2186648444 wait complete. 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 Processor -- start 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 -- start start 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d540744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 0x7f5d540749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:59.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541a6540 con 0x7f5d54107ca0 2024-03-11T19:21:59.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541a66b0 con 0x7f5d54101cc0 2024-03-11T19:21:59.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.530+0000 7f5d5aa00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541a6870 con 0x7f5d54102680 2024-03-11T19:21:59.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d540744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:59.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:59.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d540744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:57814/0 (socket says 172.21.15.72:57814) 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53400700 1 -- 172.21.15.72:0/3317169529 learned_addr learned my addr 172.21.15.72:0/3317169529 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 -- 172.21.15.72:0/3317169529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 msgr2=0x7f5d540744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d540744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 -- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 msgr2=0x7f5d540749f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 0x7f5d540749f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 -- 172.21.15.72:0/3317169529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440095f0 con 0x7f5d54101cc0 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.531+0000 7f5d53e00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54073f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d440098e0 tx=0x7f5d44005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.532+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4401d070 con 0x7f5d54101cc0 2024-03-11T19:21:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.532+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d541a6af0 con 0x7f5d54101cc0 2024-03-11T19:21:59.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.532+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d44003680 con 0x7f5d54101cc0 2024-03-11T19:21:59.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.532+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400fa30 con 0x7f5d54101cc0 2024-03-11T19:21:59.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.532+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541a6f10 con 0x7f5d54101cc0 2024-03-11T19:21:59.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.533+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5410a180 con 0x7f5d54101cc0 2024-03-11T19:21:59.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.535+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5d4400b790 con 0x7f5d54101cc0 2024-03-11T19:21:59.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.536+0000 7f5d50c00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 0x7f5d380a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:21:59.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.536+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f5d440ce290 con 0x7f5d54101cc0 2024-03-11T19:21:59.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.536+0000 7f5d53400700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 0x7f5d380a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:21:59.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.537+0000 7f5d53400700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 0x7f5d380a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d54079410 tx=0x7f5d40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:21:59.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.540+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d4405cd20 con 0x7f5d54101cc0 2024-03-11T19:21:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:21:59 smithi106 ceph-mon[35889]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:21:59 smithi143 ceph-mon[36013]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:21:59.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.754+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d54111e20 con 0x7f5d380a1ac0 2024-03-11T19:21:59.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.783+0000 7f5d50c00700 1 -- 172.21.15.72:0/3317169529 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340800 (secure 0 0 0) 0x7f5d54111e20 con 0x7f5d380a1ac0 2024-03-11T19:21:59.790 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:21:59.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 msgr2=0x7f5d380a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 0x7f5d380a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d54079410 tx=0x7f5d40009260 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 msgr2=0x7f5d54073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54073f70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d440098e0 tx=0x7f5d44005280 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 shutdown_connections 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5d380a1ac0 0x7f5d380a3f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d54107ca0 0x7f5d540749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d54101cc0 0x7f5d54073f70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 --2- 172.21.15.72:0/3317169529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d54102680 0x7f5d540744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 >> 172.21.15.72:0/3317169529 conn(0x7f5d54078da0 msgr2=0x7f5d54100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 shutdown_connections 2024-03-11T19:21:59.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:21:59.792+0000 7f5d5aa00700 1 -- 172.21.15.72:0/3317169529 wait complete. 2024-03-11T19:21:59.798 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-03-11T19:22:00.735 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-03-11T19:21:58.115661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":769,"num_read_kb":508,"num_write":353,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370084,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571740,"statfs":{"total":1151940427776,"available":1151561461760,"internally_reserved":0,"allocated":12836864,"data_stored":8225492,"data_compressed":431934,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006016"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481428+0000","last_change":"2024-03-11T19:19:42.386329+0000","last_active":"2024-03-11T19:21:32.481428+0000","last_peered":"2024-03-11T19:21:32.481428+0000","last_clean":"2024-03-11T19:21:32.481428+0000","last_became_active":"2024-03-11T19:19:42.386041+0000","last_became_peered":"2024-03-11T19:19:42.386041+0000","last_unstale":"2024-03-11T19:21:32.481428+0000","last_undegraded":"2024-03-11T19:21:32.481428+0000","last_fullsized":"2024-03-11T19:21:32.481428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:05:38.258840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481854+0000","last_change":"2024-03-11T19:19:42.381508+0000","last_active":"2024-03-11T19:21:32.481854+0000","last_peered":"2024-03-11T19:21:32.481854+0000","last_clean":"2024-03-11T19:21:32.481854+0000","last_became_active":"2024-03-11T19:19:42.378340+0000","last_became_peered":"2024-03-11T19:19:42.378340+0000","last_unstale":"2024-03-11T19:21:32.481854+0000","last_undegraded":"2024-03-11T19:21:32.481854+0000","last_fullsized":"2024-03-11T19:21:32.481854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:31:06.472488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481293+0000","last_change":"2024-03-11T19:19:42.377904+0000","last_active":"2024-03-11T19:21:32.481293+0000","last_peered":"2024-03-11T19:21:32.481293+0000","last_clean":"2024-03-11T19:21:32.481293+0000","last_became_active":"2024-03-11T19:19:42.376279+0000","last_became_peered":"2024-03-11T19:19:42.376279+0000","last_unstale":"2024-03-11T19:21:32.481293+0000","last_undegraded":"2024-03-11T19:21:32.481293+0000","last_fullsized":"2024-03-11T19:21:32.481293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:52.685543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.325869+0000","last_change":"2024-03-11T19:21:32.324870+0000","last_active":"2024-03-11T19:21:32.325869+0000","last_peered":"2024-03-11T19:21:32.325869+0000","last_clean":"2024-03-11T19:21:32.325869+0000","last_became_active":"2024-03-11T19:21:32.323768+0000","last_became_peered":"2024-03-11T19:21:32.323768+0000","last_unstale":"2024-03-11T19:21:32.325869+0000","last_undegraded":"2024-03-11T19:21:32.325869+0000","last_fullsized":"2024-03-11T19:21:32.325869+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:18:47.489312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322158+0000","last_change":"2024-03-11T19:19:42.380979+0000","last_active":"2024-03-11T19:21:32.322158+0000","last_peered":"2024-03-11T19:21:32.322158+0000","last_clean":"2024-03-11T19:21:32.322158+0000","last_became_active":"2024-03-11T19:19:42.380653+0000","last_became_peered":"2024-03-11T19:19:42.380653+0000","last_unstale":"2024-03-11T19:21:32.322158+0000","last_undegraded":"2024-03-11T19:21:32.322158+0000","last_fullsized":"2024-03-11T19:21:32.322158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:39.534955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313025+0000","last_change":"2024-03-11T19:19:42.378182+0000","last_active":"2024-03-11T19:21:32.313025+0000","last_peered":"2024-03-11T19:21:32.313025+0000","last_clean":"2024-03-11T19:21:32.313025+0000","last_became_active":"2024-03-11T19:19:42.377810+0000","last_became_peered":"2024-03-11T19:19:42.377810+0000","last_unstale":"2024-03-11T19:21:32.313025+0000","last_undegraded":"2024-03-11T19:21:32.313025+0000","last_fullsized":"2024-03-11T19:21:32.313025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:37:20.498630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480620+0000","last_change":"2024-03-11T19:19:42.370872+0000","last_active":"2024-03-11T19:21:32.480620+0000","last_peered":"2024-03-11T19:21:32.480620+0000","last_clean":"2024-03-11T19:21:32.480620+0000","last_became_active":"2024-03-11T19:19:42.370543+0000","last_became_peered":"2024-03-11T19:19:42.370543+0000","last_unstale":"2024-03-11T19:21:32.480620+0000","last_undegraded":"2024-03-11T19:21:32.480620+0000","last_fullsized":"2024-03-11T19:21:32.480620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:40:48.483035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317106+0000","last_change":"2024-03-11T19:20:31.330881+0000","last_active":"2024-03-11T19:21:32.317106+0000","last_peered":"2024-03-11T19:21:32.317106+0000","last_clean":"2024-03-11T19:21:32.317106+0000","last_became_active":"2024-03-11T19:20:31.328780+0000","last_became_peered":"2024-03-11T19:20:31.328780+0000","last_unstale":"2024-03-11T19:21:32.317106+0000","last_undegraded":"2024-03-11T19:21:32.317106+0000","last_fullsized":"2024-03-11T19:21:32.317106+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:18.157119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481834+0000","last_change":"2024-03-11T19:19:42.368501+0000","last_active":"2024-03-11T19:21:32.481834+0000","last_peered":"2024-03-11T19:21:32.481834+0000","last_clean":"2024-03-11T19:21:32.481834+0000","last_became_active":"2024-03-11T19:19:42.368155+0000","last_became_peered":"2024-03-11T19:19:42.368155+0000","last_unstale":"2024-03-11T19:21:32.481834+0000","last_undegraded":"2024-03-11T19:21:32.481834+0000","last_fullsized":"2024-03-11T19:21:32.481834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:55:59.244376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313368+0000","last_change":"2024-03-11T19:19:42.365113+0000","last_active":"2024-03-11T19:21:32.313368+0000","last_peered":"2024-03-11T19:21:32.313368+0000","last_clean":"2024-03-11T19:21:32.313368+0000","last_became_active":"2024-03-11T19:19:42.364710+0000","last_became_peered":"2024-03-11T19:19:42.364710+0000","last_unstale":"2024-03-11T19:21:32.313368+0000","last_undegraded":"2024-03-11T19:21:32.313368+0000","last_fullsized":"2024-03-11T19:21:32.313368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:28.999688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481609+0000","last_change":"2024-03-11T19:19:42.365977+0000","last_active":"2024-03-11T19:21:32.481609+0000","last_peered":"2024-03-11T19:21:32.481609+0000","last_clean":"2024-03-11T19:21:32.481609+0000","last_became_active":"2024-03-11T19:19:42.365220+0000","last_became_peered":"2024-03-11T19:19:42.365220+0000","last_unstale":"2024-03-11T19:21:32.481609+0000","last_undegraded":"2024-03-11T19:21:32.481609+0000","last_fullsized":"2024-03-11T19:21:32.481609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:45.707973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.319030+0000","last_change":"2024-03-11T19:20:31.325750+0000","last_active":"2024-03-11T19:21:32.319030+0000","last_peered":"2024-03-11T19:21:32.319030+0000","last_clean":"2024-03-11T19:21:32.319030+0000","last_became_active":"2024-03-11T19:20:31.324671+0000","last_became_peered":"2024-03-11T19:20:31.324671+0000","last_unstale":"2024-03-11T19:21:32.319030+0000","last_undegraded":"2024-03-11T19:21:32.319030+0000","last_fullsized":"2024-03-11T19:21:32.319030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:01:17.238573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413060+0000","last_change":"2024-03-11T19:19:42.374433+0000","last_active":"2024-03-11T19:21:32.413060+0000","last_peered":"2024-03-11T19:21:32.413060+0000","last_clean":"2024-03-11T19:21:32.413060+0000","last_became_active":"2024-03-11T19:19:42.371924+0000","last_became_peered":"2024-03-11T19:19:42.371924+0000","last_unstale":"2024-03-11T19:21:32.413060+0000","last_undegraded":"2024-03-11T19:21:32.413060+0000","last_fullsized":"2024-03-11T19:21:32.413060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:37.515381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481181+0000","last_change":"2024-03-11T19:19:42.378885+0000","last_active":"2024-03-11T19:21:32.481181+0000","last_peered":"2024-03-11T19:21:32.481181+0000","last_clean":"2024-03-11T19:21:32.481181+0000","last_became_active":"2024-03-11T19:19:42.378505+0000","last_became_peered":"2024-03-11T19:19:42.378505+0000","last_unstale":"2024-03-11T19:21:32.481181+0000","last_undegraded":"2024-03-11T19:21:32.481181+0000","last_fullsized":"2024-03-11T19:21:32.481181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:12.380431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481048+0000","last_change":"2024-03-11T19:19:42.377500+0000","last_active":"2024-03-11T19:21:32.481048+0000","last_peered":"2024-03-11T19:21:32.481048+0000","last_clean":"2024-03-11T19:21:32.481048+0000","last_became_active":"2024-03-11T19:19:42.377083+0000","last_became_peered":"2024-03-11T19:19:42.377083+0000","last_unstale":"2024-03-11T19:21:32.481048+0000","last_undegraded":"2024-03-11T19:21:32.481048+0000","last_fullsized":"2024-03-11T19:21:32.481048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:02:56.146071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480847+0000","last_change":"2024-03-11T19:19:42.375809+0000","last_active":"2024-03-11T19:21:32.480847+0000","last_peered":"2024-03-11T19:21:32.480847+0000","last_clean":"2024-03-11T19:21:32.480847+0000","last_became_active":"2024-03-11T19:19:42.375422+0000","last_became_peered":"2024-03-11T19:19:42.375422+0000","last_unstale":"2024-03-11T19:21:32.480847+0000","last_undegraded":"2024-03-11T19:21:32.480847+0000","last_fullsized":"2024-03-11T19:21:32.480847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:17:21.344587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481266+0000","last_change":"2024-03-11T19:19:42.381932+0000","last_active":"2024-03-11T19:21:32.481266+0000","last_peered":"2024-03-11T19:21:32.481266+0000","last_clean":"2024-03-11T19:21:32.481266+0000","last_became_active":"2024-03-11T19:19:42.381591+0000","last_became_peered":"2024-03-11T19:19:42.381591+0000","last_unstale":"2024-03-11T19:21:32.481266+0000","last_undegraded":"2024-03-11T19:21:32.481266+0000","last_fullsized":"2024-03-11T19:21:32.481266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:03.619050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318747+0000","last_change":"2024-03-11T19:19:42.374924+0000","last_active":"2024-03-11T19:21:32.318747+0000","last_peered":"2024-03-11T19:21:32.318747+0000","last_clean":"2024-03-11T19:21:32.318747+0000","last_became_active":"2024-03-11T19:19:42.374094+0000","last_became_peered":"2024-03-11T19:19:42.374094+0000","last_unstale":"2024-03-11T19:21:32.318747+0000","last_undegraded":"2024-03-11T19:21:32.318747+0000","last_fullsized":"2024-03-11T19:21:32.318747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:15.327665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313901+0000","last_change":"2024-03-11T19:19:42.381807+0000","last_active":"2024-03-11T19:21:32.313901+0000","last_peered":"2024-03-11T19:21:32.313901+0000","last_clean":"2024-03-11T19:21:32.313901+0000","last_became_active":"2024-03-11T19:19:42.381056+0000","last_became_peered":"2024-03-11T19:19:42.381056+0000","last_unstale":"2024-03-11T19:21:32.313901+0000","last_undegraded":"2024-03-11T19:21:32.313901+0000","last_fullsized":"2024-03-11T19:21:32.313901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:58:00.221003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313259+0000","last_change":"2024-03-11T19:19:46.825875+0000","last_active":"2024-03-11T19:21:32.313259+0000","last_peered":"2024-03-11T19:21:32.313259+0000","last_clean":"2024-03-11T19:21:32.313259+0000","last_became_active":"2024-03-11T19:19:46.825528+0000","last_became_peered":"2024-03-11T19:19:46.825528+0000","last_unstale":"2024-03-11T19:21:32.313259+0000","last_undegraded":"2024-03-11T19:21:32.313259+0000","last_fullsized":"2024-03-11T19:21:32.313259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:32:38.667775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318722+0000","last_change":"2024-03-11T19:19:41.393633+0000","last_active":"2024-03-11T19:21:32.318722+0000","last_peered":"2024-03-11T19:21:32.318722+0000","last_clean":"2024-03-11T19:21:32.318722+0000","last_became_active":"2024-03-11T19:19:41.389865+0000","last_became_peered":"2024-03-11T19:19:41.389865+0000","last_unstale":"2024-03-11T19:21:32.318722+0000","last_undegraded":"2024-03-11T19:21:32.318722+0000","last_fullsized":"2024-03-11T19:21:32.318722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:29.251688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315501+0000","last_change":"2024-03-11T19:19:42.392393+0000","last_active":"2024-03-11T19:21:32.315501+0000","last_peered":"2024-03-11T19:21:32.315501+0000","last_clean":"2024-03-11T19:21:32.315501+0000","last_became_active":"2024-03-11T19:19:42.390724+0000","last_became_peered":"2024-03-11T19:19:42.390724+0000","last_unstale":"2024-03-11T19:21:32.315501+0000","last_undegraded":"2024-03-11T19:21:32.315501+0000","last_fullsized":"2024-03-11T19:21:32.315501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:49.410831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416082+0000","last_change":"2024-03-11T19:19:46.817231+0000","last_active":"2024-03-11T19:21:32.416082+0000","last_peered":"2024-03-11T19:21:32.416082+0000","last_clean":"2024-03-11T19:21:32.416082+0000","last_became_active":"2024-03-11T19:19:46.816818+0000","last_became_peered":"2024-03-11T19:19:46.816818+0000","last_unstale":"2024-03-11T19:21:32.416082+0000","last_undegraded":"2024-03-11T19:21:32.416082+0000","last_fullsized":"2024-03-11T19:21:32.416082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:49.562628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315668+0000","last_change":"2024-03-11T19:19:41.401029+0000","last_active":"2024-03-11T19:21:32.315668+0000","last_peered":"2024-03-11T19:21:32.315668+0000","last_clean":"2024-03-11T19:21:32.315668+0000","last_became_active":"2024-03-11T19:19:41.400659+0000","last_became_peered":"2024-03-11T19:19:41.400659+0000","last_unstale":"2024-03-11T19:21:32.315668+0000","last_undegraded":"2024-03-11T19:21:32.315668+0000","last_fullsized":"2024-03-11T19:21:32.315668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:07.903543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481188+0000","last_change":"2024-03-11T19:19:42.380714+0000","last_active":"2024-03-11T19:21:32.481188+0000","last_peered":"2024-03-11T19:21:32.481188+0000","last_clean":"2024-03-11T19:21:32.481188+0000","last_became_active":"2024-03-11T19:19:42.380064+0000","last_became_peered":"2024-03-11T19:19:42.380064+0000","last_unstale":"2024-03-11T19:21:32.481188+0000","last_undegraded":"2024-03-11T19:21:32.481188+0000","last_fullsized":"2024-03-11T19:21:32.481188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:47.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416161+0000","last_change":"2024-03-11T19:19:46.815643+0000","last_active":"2024-03-11T19:21:32.416161+0000","last_peered":"2024-03-11T19:21:32.416161+0000","last_clean":"2024-03-11T19:21:32.416161+0000","last_became_active":"2024-03-11T19:19:46.815222+0000","last_became_peered":"2024-03-11T19:19:46.815222+0000","last_unstale":"2024-03-11T19:21:32.416161+0000","last_undegraded":"2024-03-11T19:21:32.416161+0000","last_fullsized":"2024-03-11T19:21:32.416161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:44:24.625495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322598+0000","last_change":"2024-03-11T19:19:41.397810+0000","last_active":"2024-03-11T19:21:32.322598+0000","last_peered":"2024-03-11T19:21:32.322598+0000","last_clean":"2024-03-11T19:21:32.322598+0000","last_became_active":"2024-03-11T19:19:41.396726+0000","last_became_peered":"2024-03-11T19:19:41.396726+0000","last_unstale":"2024-03-11T19:21:32.322598+0000","last_undegraded":"2024-03-11T19:21:32.322598+0000","last_fullsized":"2024-03-11T19:21:32.322598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:26.328409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481185+0000","last_change":"2024-03-11T19:19:42.360595+0000","last_active":"2024-03-11T19:21:32.481185+0000","last_peered":"2024-03-11T19:21:32.481185+0000","last_clean":"2024-03-11T19:21:32.481185+0000","last_became_active":"2024-03-11T19:19:42.360230+0000","last_became_peered":"2024-03-11T19:19:42.360230+0000","last_unstale":"2024-03-11T19:21:32.481185+0000","last_undegraded":"2024-03-11T19:21:32.481185+0000","last_fullsized":"2024-03-11T19:21:32.481185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:45.480391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416111+0000","last_change":"2024-03-11T19:19:46.817442+0000","last_active":"2024-03-11T19:21:32.416111+0000","last_peered":"2024-03-11T19:21:32.416111+0000","last_clean":"2024-03-11T19:21:32.416111+0000","last_became_active":"2024-03-11T19:19:46.816944+0000","last_became_peered":"2024-03-11T19:19:46.816944+0000","last_unstale":"2024-03-11T19:21:32.416111+0000","last_undegraded":"2024-03-11T19:21:32.416111+0000","last_fullsized":"2024-03-11T19:21:32.416111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:05:43.783450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322442+0000","last_change":"2024-03-11T19:19:41.389608+0000","last_active":"2024-03-11T19:21:32.322442+0000","last_peered":"2024-03-11T19:21:32.322442+0000","last_clean":"2024-03-11T19:21:32.322442+0000","last_became_active":"2024-03-11T19:19:41.389281+0000","last_became_peered":"2024-03-11T19:19:41.389281+0000","last_unstale":"2024-03-11T19:21:32.322442+0000","last_undegraded":"2024-03-11T19:21:32.322442+0000","last_fullsized":"2024-03-11T19:21:32.322442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:46.213494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357364+0000","last_change":"2024-03-11T19:19:42.371444+0000","last_active":"2024-03-11T19:21:32.357364+0000","last_peered":"2024-03-11T19:21:32.357364+0000","last_clean":"2024-03-11T19:21:32.357364+0000","last_became_active":"2024-03-11T19:19:42.371129+0000","last_became_peered":"2024-03-11T19:19:42.371129+0000","last_unstale":"2024-03-11T19:21:32.357364+0000","last_undegraded":"2024-03-11T19:21:32.357364+0000","last_fullsized":"2024-03-11T19:21:32.357364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:04:07.449453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313758+0000","last_change":"2024-03-11T19:19:46.805616+0000","last_active":"2024-03-11T19:21:32.313758+0000","last_peered":"2024-03-11T19:21:32.313758+0000","last_clean":"2024-03-11T19:21:32.313758+0000","last_became_active":"2024-03-11T19:19:46.805069+0000","last_became_peered":"2024-03-11T19:19:46.805069+0000","last_unstale":"2024-03-11T19:21:32.313758+0000","last_undegraded":"2024-03-11T19:21:32.313758+0000","last_fullsized":"2024-03-11T19:21:32.313758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:47:23.197283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:52.385870+0000","last_change":"2024-03-11T19:19:41.392375+0000","last_active":"2024-03-11T19:21:52.385870+0000","last_peered":"2024-03-11T19:21:52.385870+0000","last_clean":"2024-03-11T19:21:52.385870+0000","last_became_active":"2024-03-11T19:19:41.392057+0000","last_became_peered":"2024-03-11T19:19:41.392057+0000","last_unstale":"2024-03-11T19:21:52.385870+0000","last_undegraded":"2024-03-11T19:21:52.385870+0000","last_fullsized":"2024-03-11T19:21:52.385870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:12.842671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322166+0000","last_change":"2024-03-11T19:19:42.373255+0000","last_active":"2024-03-11T19:21:32.322166+0000","last_peered":"2024-03-11T19:21:32.322166+0000","last_clean":"2024-03-11T19:21:32.322166+0000","last_became_active":"2024-03-11T19:19:42.372900+0000","last_became_peered":"2024-03-11T19:19:42.372900+0000","last_unstale":"2024-03-11T19:21:32.322166+0000","last_undegraded":"2024-03-11T19:21:32.322166+0000","last_fullsized":"2024-03-11T19:21:32.322166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:56:45.517189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313365+0000","last_change":"2024-03-11T19:20:31.363743+0000","last_active":"2024-03-11T19:21:32.313365+0000","last_peered":"2024-03-11T19:21:32.313365+0000","last_clean":"2024-03-11T19:21:32.313365+0000","last_became_active":"2024-03-11T19:20:31.363255+0000","last_became_peered":"2024-03-11T19:20:31.363255+0000","last_unstale":"2024-03-11T19:21:32.313365+0000","last_undegraded":"2024-03-11T19:21:32.313365+0000","last_fullsized":"2024-03-11T19:21:32.313365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:51:05.796735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.312970+0000","last_change":"2024-03-11T19:19:41.385591+0000","last_active":"2024-03-11T19:21:32.312970+0000","last_peered":"2024-03-11T19:21:32.312970+0000","last_clean":"2024-03-11T19:21:32.312970+0000","last_became_active":"2024-03-11T19:19:41.385236+0000","last_became_peered":"2024-03-11T19:19:41.385236+0000","last_unstale":"2024-03-11T19:21:32.312970+0000","last_undegraded":"2024-03-11T19:21:32.312970+0000","last_fullsized":"2024-03-11T19:21:32.312970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:55:55.558948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357301+0000","last_change":"2024-03-11T19:19:42.380934+0000","last_active":"2024-03-11T19:21:32.357301+0000","last_peered":"2024-03-11T19:21:32.357301+0000","last_clean":"2024-03-11T19:21:32.357301+0000","last_became_active":"2024-03-11T19:19:42.380602+0000","last_became_peered":"2024-03-11T19:19:42.380602+0000","last_unstale":"2024-03-11T19:21:32.357301+0000","last_undegraded":"2024-03-11T19:21:32.357301+0000","last_fullsized":"2024-03-11T19:21:32.357301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:58:26.813552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481285+0000","last_change":"2024-03-11T19:19:46.814227+0000","last_active":"2024-03-11T19:21:32.481285+0000","last_peered":"2024-03-11T19:21:32.481285+0000","last_clean":"2024-03-11T19:21:32.481285+0000","last_became_active":"2024-03-11T19:19:46.813874+0000","last_became_peered":"2024-03-11T19:19:46.813874+0000","last_unstale":"2024-03-11T19:21:32.481285+0000","last_undegraded":"2024-03-11T19:21:32.481285+0000","last_fullsized":"2024-03-11T19:21:32.481285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:21.493313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412609+0000","last_change":"2024-03-11T19:19:41.392245+0000","last_active":"2024-03-11T19:21:32.412609+0000","last_peered":"2024-03-11T19:21:32.412609+0000","last_clean":"2024-03-11T19:21:32.412609+0000","last_became_active":"2024-03-11T19:19:41.391928+0000","last_became_peered":"2024-03-11T19:19:41.391928+0000","last_unstale":"2024-03-11T19:21:32.412609+0000","last_undegraded":"2024-03-11T19:21:32.412609+0000","last_fullsized":"2024-03-11T19:21:32.412609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:20.556826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314283+0000","last_change":"2024-03-11T19:19:42.389431+0000","last_active":"2024-03-11T19:21:32.314283+0000","last_peered":"2024-03-11T19:21:32.314283+0000","last_clean":"2024-03-11T19:21:32.314283+0000","last_became_active":"2024-03-11T19:19:42.387885+0000","last_became_peered":"2024-03-11T19:19:42.387885+0000","last_unstale":"2024-03-11T19:21:32.314283+0000","last_undegraded":"2024-03-11T19:21:32.314283+0000","last_fullsized":"2024-03-11T19:21:32.314283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:10:43.753079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481707+0000","last_change":"2024-03-11T19:19:46.823474+0000","last_active":"2024-03-11T19:21:32.481707+0000","last_peered":"2024-03-11T19:21:32.481707+0000","last_clean":"2024-03-11T19:21:32.481707+0000","last_became_active":"2024-03-11T19:19:46.823008+0000","last_became_peered":"2024-03-11T19:19:46.823008+0000","last_unstale":"2024-03-11T19:21:32.481707+0000","last_undegraded":"2024-03-11T19:21:32.481707+0000","last_fullsized":"2024-03-11T19:21:32.481707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:28.381809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356737+0000","last_change":"2024-03-11T19:19:41.395332+0000","last_active":"2024-03-11T19:21:32.356737+0000","last_peered":"2024-03-11T19:21:32.356737+0000","last_clean":"2024-03-11T19:21:32.356737+0000","last_became_active":"2024-03-11T19:19:41.394989+0000","last_became_peered":"2024-03-11T19:19:41.394989+0000","last_unstale":"2024-03-11T19:21:32.356737+0000","last_undegraded":"2024-03-11T19:21:32.356737+0000","last_fullsized":"2024-03-11T19:21:32.356737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:57:31.486891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318863+0000","last_change":"2024-03-11T19:19:42.393223+0000","last_active":"2024-03-11T19:21:32.318863+0000","last_peered":"2024-03-11T19:21:32.318863+0000","last_clean":"2024-03-11T19:21:32.318863+0000","last_became_active":"2024-03-11T19:19:42.392787+0000","last_became_peered":"2024-03-11T19:19:42.392787+0000","last_unstale":"2024-03-11T19:21:32.318863+0000","last_undegraded":"2024-03-11T19:21:32.318863+0000","last_fullsized":"2024-03-11T19:21:32.318863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:32:50.693896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314245+0000","last_change":"2024-03-11T19:19:46.808305+0000","last_active":"2024-03-11T19:21:32.314245+0000","last_peered":"2024-03-11T19:21:32.314245+0000","last_clean":"2024-03-11T19:21:32.314245+0000","last_became_active":"2024-03-11T19:19:46.807816+0000","last_became_peered":"2024-03-11T19:19:46.807816+0000","last_unstale":"2024-03-11T19:21:32.314245+0000","last_undegraded":"2024-03-11T19:21:32.314245+0000","last_fullsized":"2024-03-11T19:21:32.314245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:55:43.716762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":136,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:55.389346+0000","last_change":"2024-03-11T19:19:41.389335+0000","last_active":"2024-03-11T19:21:55.389346+0000","last_peered":"2024-03-11T19:21:55.389346+0000","last_clean":"2024-03-11T19:21:55.389346+0000","last_became_active":"2024-03-11T19:19:41.389023+0000","last_became_peered":"2024-03-11T19:19:41.389023+0000","last_unstale":"2024-03-11T19:21:55.389346+0000","last_undegraded":"2024-03-11T19:21:55.389346+0000","last_fullsized":"2024-03-11T19:21:55.389346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:12:59.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481552+0000","last_change":"2024-03-11T19:19:42.378379+0000","last_active":"2024-03-11T19:21:32.481552+0000","last_peered":"2024-03-11T19:21:32.481552+0000","last_clean":"2024-03-11T19:21:32.481552+0000","last_became_active":"2024-03-11T19:19:42.378033+0000","last_became_peered":"2024-03-11T19:19:42.378033+0000","last_unstale":"2024-03-11T19:21:32.481552+0000","last_undegraded":"2024-03-11T19:21:32.481552+0000","last_fullsized":"2024-03-11T19:21:32.481552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:24.154420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317132+0000","last_change":"2024-03-11T19:19:46.809464+0000","last_active":"2024-03-11T19:21:32.317132+0000","last_peered":"2024-03-11T19:21:32.317132+0000","last_clean":"2024-03-11T19:21:32.317132+0000","last_became_active":"2024-03-11T19:19:46.808955+0000","last_became_peered":"2024-03-11T19:19:46.808955+0000","last_unstale":"2024-03-11T19:21:32.317132+0000","last_undegraded":"2024-03-11T19:21:32.317132+0000","last_fullsized":"2024-03-11T19:21:32.317132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:42.088992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481875+0000","last_change":"2024-03-11T19:19:41.395619+0000","last_active":"2024-03-11T19:21:32.481875+0000","last_peered":"2024-03-11T19:21:32.481875+0000","last_clean":"2024-03-11T19:21:32.481875+0000","last_became_active":"2024-03-11T19:19:41.394658+0000","last_became_peered":"2024-03-11T19:19:41.394658+0000","last_unstale":"2024-03-11T19:21:32.481875+0000","last_undegraded":"2024-03-11T19:21:32.481875+0000","last_fullsized":"2024-03-11T19:21:32.481875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:10:39.020508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481332+0000","last_change":"2024-03-11T19:19:42.380416+0000","last_active":"2024-03-11T19:21:32.481332+0000","last_peered":"2024-03-11T19:21:32.481332+0000","last_clean":"2024-03-11T19:21:32.481332+0000","last_became_active":"2024-03-11T19:19:42.379625+0000","last_became_peered":"2024-03-11T19:19:42.379625+0000","last_unstale":"2024-03-11T19:21:32.481332+0000","last_undegraded":"2024-03-11T19:21:32.481332+0000","last_fullsized":"2024-03-11T19:21:32.481332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:45:45.374211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416320+0000","last_change":"2024-03-11T19:19:46.808005+0000","last_active":"2024-03-11T19:21:32.416320+0000","last_peered":"2024-03-11T19:21:32.416320+0000","last_clean":"2024-03-11T19:21:32.416320+0000","last_became_active":"2024-03-11T19:19:46.807549+0000","last_became_peered":"2024-03-11T19:19:46.807549+0000","last_unstale":"2024-03-11T19:21:32.416320+0000","last_undegraded":"2024-03-11T19:21:32.416320+0000","last_fullsized":"2024-03-11T19:21:32.416320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:17.171846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357344+0000","last_change":"2024-03-11T19:19:41.398755+0000","last_active":"2024-03-11T19:21:32.357344+0000","last_peered":"2024-03-11T19:21:32.357344+0000","last_clean":"2024-03-11T19:21:32.357344+0000","last_became_active":"2024-03-11T19:19:41.398406+0000","last_became_peered":"2024-03-11T19:19:41.398406+0000","last_unstale":"2024-03-11T19:21:32.357344+0000","last_undegraded":"2024-03-11T19:21:32.357344+0000","last_fullsized":"2024-03-11T19:21:32.357344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:37.793328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481540+0000","last_change":"2024-03-11T19:19:42.364757+0000","last_active":"2024-03-11T19:21:32.481540+0000","last_peered":"2024-03-11T19:21:32.481540+0000","last_clean":"2024-03-11T19:21:32.481540+0000","last_became_active":"2024-03-11T19:19:42.364432+0000","last_became_peered":"2024-03-11T19:19:42.364432+0000","last_unstale":"2024-03-11T19:21:32.481540+0000","last_undegraded":"2024-03-11T19:21:32.481540+0000","last_fullsized":"2024-03-11T19:21:32.481540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:29.226200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317967+0000","last_change":"2024-03-11T19:19:46.809300+0000","last_active":"2024-03-11T19:21:32.317967+0000","last_peered":"2024-03-11T19:21:32.317967+0000","last_clean":"2024-03-11T19:21:32.317967+0000","last_became_active":"2024-03-11T19:19:46.807327+0000","last_became_peered":"2024-03-11T19:19:46.807327+0000","last_unstale":"2024-03-11T19:21:32.317967+0000","last_undegraded":"2024-03-11T19:21:32.317967+0000","last_fullsized":"2024-03-11T19:21:32.317967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:37:54.082436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:56.390551+0000","last_change":"2024-03-11T19:19:41.392270+0000","last_active":"2024-03-11T19:21:56.390551+0000","last_peered":"2024-03-11T19:21:56.390551+0000","last_clean":"2024-03-11T19:21:56.390551+0000","last_became_active":"2024-03-11T19:19:41.391559+0000","last_became_peered":"2024-03-11T19:19:41.391559+0000","last_unstale":"2024-03-11T19:21:56.390551+0000","last_undegraded":"2024-03-11T19:21:56.390551+0000","last_fullsized":"2024-03-11T19:21:56.390551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:45.655610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":167,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322449+0000","last_change":"2024-03-11T19:19:42.368323+0000","last_active":"2024-03-11T19:21:32.322449+0000","last_peered":"2024-03-11T19:21:32.322449+0000","last_clean":"2024-03-11T19:21:32.322449+0000","last_became_active":"2024-03-11T19:19:42.367973+0000","last_became_peered":"2024-03-11T19:19:42.367973+0000","last_unstale":"2024-03-11T19:21:32.322449+0000","last_undegraded":"2024-03-11T19:21:32.322449+0000","last_fullsized":"2024-03-11T19:21:32.322449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:11.573348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357646+0000","last_change":"2024-03-11T19:19:46.822255+0000","last_active":"2024-03-11T19:21:32.357646+0000","last_peered":"2024-03-11T19:21:32.357646+0000","last_clean":"2024-03-11T19:21:32.357646+0000","last_became_active":"2024-03-11T19:19:46.821306+0000","last_became_peered":"2024-03-11T19:19:46.821306+0000","last_unstale":"2024-03-11T19:21:32.357646+0000","last_undegraded":"2024-03-11T19:21:32.357646+0000","last_fullsized":"2024-03-11T19:21:32.357646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:14:22.353676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313431+0000","last_change":"2024-03-11T19:19:41.397723+0000","last_active":"2024-03-11T19:21:32.313431+0000","last_peered":"2024-03-11T19:21:32.313431+0000","last_clean":"2024-03-11T19:21:32.313431+0000","last_became_active":"2024-03-11T19:19:41.397394+0000","last_became_peered":"2024-03-11T19:19:41.397394+0000","last_unstale":"2024-03-11T19:21:32.313431+0000","last_undegraded":"2024-03-11T19:21:32.313431+0000","last_fullsized":"2024-03-11T19:21:32.313431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:48:42.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412447+0000","last_change":"2024-03-11T19:19:42.389254+0000","last_active":"2024-03-11T19:21:32.412447+0000","last_peered":"2024-03-11T19:21:32.412447+0000","last_clean":"2024-03-11T19:21:32.412447+0000","last_became_active":"2024-03-11T19:19:42.388861+0000","last_became_peered":"2024-03-11T19:19:42.388861+0000","last_unstale":"2024-03-11T19:21:32.412447+0000","last_undegraded":"2024-03-11T19:21:32.412447+0000","last_fullsized":"2024-03-11T19:21:32.412447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:03.308802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481035+0000","last_change":"2024-03-11T19:19:46.821667+0000","last_active":"2024-03-11T19:21:32.481035+0000","last_peered":"2024-03-11T19:21:32.481035+0000","last_clean":"2024-03-11T19:21:32.481035+0000","last_became_active":"2024-03-11T19:19:46.821238+0000","last_became_peered":"2024-03-11T19:19:46.821238+0000","last_unstale":"2024-03-11T19:21:32.481035+0000","last_undegraded":"2024-03-11T19:21:32.481035+0000","last_fullsized":"2024-03-11T19:21:32.481035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:30.997248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357102+0000","last_change":"2024-03-11T19:19:41.403277+0000","last_active":"2024-03-11T19:21:32.357102+0000","last_peered":"2024-03-11T19:21:32.357102+0000","last_clean":"2024-03-11T19:21:32.357102+0000","last_became_active":"2024-03-11T19:19:41.402875+0000","last_became_peered":"2024-03-11T19:19:41.402875+0000","last_unstale":"2024-03-11T19:21:32.357102+0000","last_undegraded":"2024-03-11T19:21:32.357102+0000","last_fullsized":"2024-03-11T19:21:32.357102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:53.937257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481455+0000","last_change":"2024-03-11T19:19:42.376244+0000","last_active":"2024-03-11T19:21:32.481455+0000","last_peered":"2024-03-11T19:21:32.481455+0000","last_clean":"2024-03-11T19:21:32.481455+0000","last_became_active":"2024-03-11T19:19:42.375896+0000","last_became_peered":"2024-03-11T19:19:42.375896+0000","last_unstale":"2024-03-11T19:21:32.481455+0000","last_undegraded":"2024-03-11T19:21:32.481455+0000","last_fullsized":"2024-03-11T19:21:32.481455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:11:46.295898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.317824+0000","last_change":"2024-03-11T19:19:46.811977+0000","last_active":"2024-03-11T19:21:32.317824+0000","last_peered":"2024-03-11T19:21:32.317824+0000","last_clean":"2024-03-11T19:21:32.317824+0000","last_became_active":"2024-03-11T19:19:46.811570+0000","last_became_peered":"2024-03-11T19:19:46.811570+0000","last_unstale":"2024-03-11T19:21:32.317824+0000","last_undegraded":"2024-03-11T19:21:32.317824+0000","last_fullsized":"2024-03-11T19:21:32.317824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:00:18.329040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":139,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:52.464616+0000","last_change":"2024-03-11T19:19:41.387779+0000","last_active":"2024-03-11T19:21:52.464616+0000","last_peered":"2024-03-11T19:21:52.464616+0000","last_clean":"2024-03-11T19:21:52.464616+0000","last_became_active":"2024-03-11T19:19:41.387424+0000","last_became_peered":"2024-03-11T19:19:41.387424+0000","last_unstale":"2024-03-11T19:21:52.464616+0000","last_undegraded":"2024-03-11T19:21:52.464616+0000","last_fullsized":"2024-03-11T19:21:52.464616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:49.936069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480660+0000","last_change":"2024-03-11T19:19:42.367664+0000","last_active":"2024-03-11T19:21:32.480660+0000","last_peered":"2024-03-11T19:21:32.480660+0000","last_clean":"2024-03-11T19:21:32.480660+0000","last_became_active":"2024-03-11T19:19:42.367343+0000","last_became_peered":"2024-03-11T19:19:42.367343+0000","last_unstale":"2024-03-11T19:21:32.480660+0000","last_undegraded":"2024-03-11T19:21:32.480660+0000","last_fullsized":"2024-03-11T19:21:32.480660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:29.215094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414451+0000","last_change":"2024-03-11T19:19:46.821832+0000","last_active":"2024-03-11T19:21:32.414451+0000","last_peered":"2024-03-11T19:21:32.414451+0000","last_clean":"2024-03-11T19:21:32.414451+0000","last_became_active":"2024-03-11T19:19:46.821183+0000","last_became_peered":"2024-03-11T19:19:46.821183+0000","last_unstale":"2024-03-11T19:21:32.414451+0000","last_undegraded":"2024-03-11T19:21:32.414451+0000","last_fullsized":"2024-03-11T19:21:32.414451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:56.293370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322016+0000","last_change":"2024-03-11T19:19:41.400790+0000","last_active":"2024-03-11T19:21:32.322016+0000","last_peered":"2024-03-11T19:21:32.322016+0000","last_clean":"2024-03-11T19:21:32.322016+0000","last_became_active":"2024-03-11T19:19:41.400456+0000","last_became_peered":"2024-03-11T19:19:41.400456+0000","last_unstale":"2024-03-11T19:21:32.322016+0000","last_undegraded":"2024-03-11T19:21:32.322016+0000","last_fullsized":"2024-03-11T19:21:32.322016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:00:18.195113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356909+0000","last_change":"2024-03-11T19:19:42.370608+0000","last_active":"2024-03-11T19:21:32.356909+0000","last_peered":"2024-03-11T19:21:32.356909+0000","last_clean":"2024-03-11T19:21:32.356909+0000","last_became_active":"2024-03-11T19:19:42.370259+0000","last_became_peered":"2024-03-11T19:19:42.370259+0000","last_unstale":"2024-03-11T19:21:32.356909+0000","last_undegraded":"2024-03-11T19:21:32.356909+0000","last_fullsized":"2024-03-11T19:21:32.356909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:50:05.778074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356874+0000","last_change":"2024-03-11T19:19:46.826437+0000","last_active":"2024-03-11T19:21:32.356874+0000","last_peered":"2024-03-11T19:21:32.356874+0000","last_clean":"2024-03-11T19:21:32.356874+0000","last_became_active":"2024-03-11T19:19:46.825964+0000","last_became_peered":"2024-03-11T19:19:46.825964+0000","last_unstale":"2024-03-11T19:21:32.356874+0000","last_undegraded":"2024-03-11T19:21:32.356874+0000","last_fullsized":"2024-03-11T19:21:32.356874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:07.970696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322748+0000","last_change":"2024-03-11T19:19:41.388747+0000","last_active":"2024-03-11T19:21:32.322748+0000","last_peered":"2024-03-11T19:21:32.322748+0000","last_clean":"2024-03-11T19:21:32.322748+0000","last_became_active":"2024-03-11T19:19:41.388408+0000","last_became_peered":"2024-03-11T19:19:41.388408+0000","last_unstale":"2024-03-11T19:21:32.322748+0000","last_undegraded":"2024-03-11T19:21:32.322748+0000","last_fullsized":"2024-03-11T19:21:32.322748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:32:20.195039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481213+0000","last_change":"2024-03-11T19:20:31.328667+0000","last_active":"2024-03-11T19:21:32.481213+0000","last_peered":"2024-03-11T19:21:32.481213+0000","last_clean":"2024-03-11T19:21:32.481213+0000","last_became_active":"2024-03-11T19:20:31.328215+0000","last_became_peered":"2024-03-11T19:20:31.328215+0000","last_unstale":"2024-03-11T19:21:32.481213+0000","last_undegraded":"2024-03-11T19:21:32.481213+0000","last_fullsized":"2024-03-11T19:21:32.481213+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:15:48.188927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322151+0000","last_change":"2024-03-11T19:19:46.934632+0000","last_active":"2024-03-11T19:21:32.322151+0000","last_peered":"2024-03-11T19:21:32.322151+0000","last_clean":"2024-03-11T19:21:32.322151+0000","last_became_active":"2024-03-11T19:19:46.934172+0000","last_became_peered":"2024-03-11T19:19:46.934172+0000","last_unstale":"2024-03-11T19:21:32.322151+0000","last_undegraded":"2024-03-11T19:21:32.322151+0000","last_fullsized":"2024-03-11T19:21:32.322151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:01:21.675154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.319056+0000","last_change":"2024-03-11T19:20:31.340137+0000","last_active":"2024-03-11T19:21:32.319056+0000","last_peered":"2024-03-11T19:21:32.319056+0000","last_clean":"2024-03-11T19:21:32.319056+0000","last_became_active":"2024-03-11T19:20:31.339299+0000","last_became_peered":"2024-03-11T19:20:31.339299+0000","last_unstale":"2024-03-11T19:21:32.319056+0000","last_undegraded":"2024-03-11T19:21:32.319056+0000","last_fullsized":"2024-03-11T19:21:32.319056+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:12:37.489161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313317+0000","last_change":"2024-03-11T19:19:42.387716+0000","last_active":"2024-03-11T19:21:32.313317+0000","last_peered":"2024-03-11T19:21:32.313317+0000","last_clean":"2024-03-11T19:21:32.313317+0000","last_became_active":"2024-03-11T19:19:42.387370+0000","last_became_peered":"2024-03-11T19:19:42.387370+0000","last_unstale":"2024-03-11T19:21:32.313317+0000","last_undegraded":"2024-03-11T19:21:32.313317+0000","last_fullsized":"2024-03-11T19:21:32.313317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:50:22.852284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413164+0000","last_change":"2024-03-11T19:19:46.827980+0000","last_active":"2024-03-11T19:21:32.413164+0000","last_peered":"2024-03-11T19:21:32.413164+0000","last_clean":"2024-03-11T19:21:32.413164+0000","last_became_active":"2024-03-11T19:19:46.827317+0000","last_became_peered":"2024-03-11T19:19:46.827317+0000","last_unstale":"2024-03-11T19:21:32.413164+0000","last_undegraded":"2024-03-11T19:21:32.413164+0000","last_fullsized":"2024-03-11T19:21:32.413164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:55:15.266791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481246+0000","last_change":"2024-03-11T19:19:41.394714+0000","last_active":"2024-03-11T19:21:32.481246+0000","last_peered":"2024-03-11T19:21:32.481246+0000","last_clean":"2024-03-11T19:21:32.481246+0000","last_became_active":"2024-03-11T19:19:41.394404+0000","last_became_peered":"2024-03-11T19:19:41.394404+0000","last_unstale":"2024-03-11T19:21:32.481246+0000","last_undegraded":"2024-03-11T19:21:32.481246+0000","last_fullsized":"2024-03-11T19:21:32.481246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:15.960037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313130+0000","last_change":"2024-03-11T19:19:42.381062+0000","last_active":"2024-03-11T19:21:32.313130+0000","last_peered":"2024-03-11T19:21:32.313130+0000","last_clean":"2024-03-11T19:21:32.313130+0000","last_became_active":"2024-03-11T19:19:42.380744+0000","last_became_peered":"2024-03-11T19:19:42.380744+0000","last_unstale":"2024-03-11T19:21:32.313130+0000","last_undegraded":"2024-03-11T19:21:32.313130+0000","last_fullsized":"2024-03-11T19:21:32.313130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:06:08.767877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414701+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:21:32.414701+0000","last_peered":"2024-03-11T19:21:32.414701+0000","last_clean":"2024-03-11T19:21:32.414701+0000","last_became_active":"2024-03-11T19:19:46.934195+0000","last_became_peered":"2024-03-11T19:19:46.934195+0000","last_unstale":"2024-03-11T19:21:32.414701+0000","last_undegraded":"2024-03-11T19:21:32.414701+0000","last_fullsized":"2024-03-11T19:21:32.414701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:33:19.268162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356902+0000","last_change":"2024-03-11T19:19:42.366940+0000","last_active":"2024-03-11T19:21:32.356902+0000","last_peered":"2024-03-11T19:21:32.356902+0000","last_clean":"2024-03-11T19:21:32.356902+0000","last_became_active":"2024-03-11T19:19:42.366628+0000","last_became_peered":"2024-03-11T19:19:42.366628+0000","last_unstale":"2024-03-11T19:21:32.356902+0000","last_undegraded":"2024-03-11T19:21:32.356902+0000","last_fullsized":"2024-03-11T19:21:32.356902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:05:37.175276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":138,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:54.468661+0000","last_change":"2024-03-11T19:19:41.389919+0000","last_active":"2024-03-11T19:21:54.468661+0000","last_peered":"2024-03-11T19:21:54.468661+0000","last_clean":"2024-03-11T19:21:54.468661+0000","last_became_active":"2024-03-11T19:19:41.389471+0000","last_became_peered":"2024-03-11T19:19:41.389471+0000","last_unstale":"2024-03-11T19:21:54.468661+0000","last_undegraded":"2024-03-11T19:21:54.468661+0000","last_fullsized":"2024-03-11T19:21:54.468661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:41.068642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481111+0000","last_change":"2024-03-11T19:20:31.339329+0000","last_active":"2024-03-11T19:21:32.481111+0000","last_peered":"2024-03-11T19:21:32.481111+0000","last_clean":"2024-03-11T19:21:32.481111+0000","last_became_active":"2024-03-11T19:20:31.328906+0000","last_became_peered":"2024-03-11T19:20:31.328906+0000","last_unstale":"2024-03-11T19:21:32.481111+0000","last_undegraded":"2024-03-11T19:21:32.481111+0000","last_fullsized":"2024-03-11T19:21:32.481111+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:31:46.707878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314799+0000","last_change":"2024-03-11T19:20:31.364729+0000","last_active":"2024-03-11T19:21:32.314799+0000","last_peered":"2024-03-11T19:21:32.314799+0000","last_clean":"2024-03-11T19:21:32.314799+0000","last_became_active":"2024-03-11T19:20:31.363713+0000","last_became_peered":"2024-03-11T19:20:31.363713+0000","last_unstale":"2024-03-11T19:21:32.314799+0000","last_undegraded":"2024-03-11T19:21:32.314799+0000","last_fullsized":"2024-03-11T19:21:32.314799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:53.829080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416389+0000","last_change":"2024-03-11T19:19:41.391501+0000","last_active":"2024-03-11T19:21:32.416389+0000","last_peered":"2024-03-11T19:21:32.416389+0000","last_clean":"2024-03-11T19:21:32.416389+0000","last_became_active":"2024-03-11T19:19:41.391148+0000","last_became_peered":"2024-03-11T19:19:41.391148+0000","last_unstale":"2024-03-11T19:21:32.416389+0000","last_undegraded":"2024-03-11T19:21:32.416389+0000","last_fullsized":"2024-03-11T19:21:32.416389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:46:18.957547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480521+0000","last_change":"2024-03-11T19:18:21.278121+0000","last_active":"2024-03-11T19:21:32.480521+0000","last_peered":"2024-03-11T19:21:32.480521+0000","last_clean":"2024-03-11T19:21:32.480521+0000","last_became_active":"2024-03-11T19:18:21.277678+0000","last_became_peered":"2024-03-11T19:18:21.277678+0000","last_unstale":"2024-03-11T19:21:32.480521+0000","last_undegraded":"2024-03-11T19:21:32.480521+0000","last_fullsized":"2024-03-11T19:21:32.480521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:14:20.582658+0000","last_clean_scrub_stamp":"2024-03-11T19:14:20.582658+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:33:19.162992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357426+0000","last_change":"2024-03-11T19:19:46.934777+0000","last_active":"2024-03-11T19:21:32.357426+0000","last_peered":"2024-03-11T19:21:32.357426+0000","last_clean":"2024-03-11T19:21:32.357426+0000","last_became_active":"2024-03-11T19:19:46.934304+0000","last_became_peered":"2024-03-11T19:19:46.934304+0000","last_unstale":"2024-03-11T19:21:32.357426+0000","last_undegraded":"2024-03-11T19:21:32.357426+0000","last_fullsized":"2024-03-11T19:21:32.357426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:34:47.392139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412388+0000","last_change":"2024-03-11T19:19:42.363283+0000","last_active":"2024-03-11T19:21:32.412388+0000","last_peered":"2024-03-11T19:21:32.412388+0000","last_clean":"2024-03-11T19:21:32.412388+0000","last_became_active":"2024-03-11T19:19:42.362930+0000","last_became_peered":"2024-03-11T19:19:42.362930+0000","last_unstale":"2024-03-11T19:21:32.412388+0000","last_undegraded":"2024-03-11T19:21:32.412388+0000","last_fullsized":"2024-03-11T19:21:32.412388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:16:36.141585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480649+0000","last_change":"2024-03-11T19:19:41.391776+0000","last_active":"2024-03-11T19:21:32.480649+0000","last_peered":"2024-03-11T19:21:32.480649+0000","last_clean":"2024-03-11T19:21:32.480649+0000","last_became_active":"2024-03-11T19:19:41.391459+0000","last_became_peered":"2024-03-11T19:19:41.391459+0000","last_unstale":"2024-03-11T19:21:32.480649+0000","last_undegraded":"2024-03-11T19:21:32.480649+0000","last_fullsized":"2024-03-11T19:21:32.480649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:36.457307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480919+0000","last_change":"2024-03-11T19:19:46.821662+0000","last_active":"2024-03-11T19:21:32.480919+0000","last_peered":"2024-03-11T19:21:32.480919+0000","last_clean":"2024-03-11T19:21:32.480919+0000","last_became_active":"2024-03-11T19:19:46.821199+0000","last_became_peered":"2024-03-11T19:19:46.821199+0000","last_unstale":"2024-03-11T19:21:32.480919+0000","last_undegraded":"2024-03-11T19:21:32.480919+0000","last_fullsized":"2024-03-11T19:21:32.480919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:08:39.658398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480994+0000","last_change":"2024-03-11T19:20:31.344359+0000","last_active":"2024-03-11T19:21:32.480994+0000","last_peered":"2024-03-11T19:21:32.480994+0000","last_clean":"2024-03-11T19:21:32.480994+0000","last_became_active":"2024-03-11T19:20:31.339153+0000","last_became_peered":"2024-03-11T19:20:31.339153+0000","last_unstale":"2024-03-11T19:21:32.480994+0000","last_undegraded":"2024-03-11T19:21:32.480994+0000","last_fullsized":"2024-03-11T19:21:32.480994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:18:35.854043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413729+0000","last_change":"2024-03-11T19:19:42.380304+0000","last_active":"2024-03-11T19:21:32.413729+0000","last_peered":"2024-03-11T19:21:32.413729+0000","last_clean":"2024-03-11T19:21:32.413729+0000","last_became_active":"2024-03-11T19:19:42.378768+0000","last_became_peered":"2024-03-11T19:19:42.378768+0000","last_unstale":"2024-03-11T19:21:32.413729+0000","last_undegraded":"2024-03-11T19:21:32.413729+0000","last_fullsized":"2024-03-11T19:21:32.413729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:31.465876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314259+0000","last_change":"2024-03-11T19:19:46.934262+0000","last_active":"2024-03-11T19:21:32.314259+0000","last_peered":"2024-03-11T19:21:32.314259+0000","last_clean":"2024-03-11T19:21:32.314259+0000","last_became_active":"2024-03-11T19:19:46.933557+0000","last_became_peered":"2024-03-11T19:19:46.933557+0000","last_unstale":"2024-03-11T19:21:32.314259+0000","last_undegraded":"2024-03-11T19:21:32.314259+0000","last_fullsized":"2024-03-11T19:21:32.314259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:29.227551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318644+0000","last_change":"2024-03-11T19:19:41.402974+0000","last_active":"2024-03-11T19:21:32.318644+0000","last_peered":"2024-03-11T19:21:32.318644+0000","last_clean":"2024-03-11T19:21:32.318644+0000","last_became_active":"2024-03-11T19:19:41.402257+0000","last_became_peered":"2024-03-11T19:19:41.402257+0000","last_unstale":"2024-03-11T19:21:32.318644+0000","last_undegraded":"2024-03-11T19:21:32.318644+0000","last_fullsized":"2024-03-11T19:21:32.318644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:00:27.136592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315380+0000","last_change":"2024-03-11T19:19:42.385331+0000","last_active":"2024-03-11T19:21:32.315380+0000","last_peered":"2024-03-11T19:21:32.315380+0000","last_clean":"2024-03-11T19:21:32.315380+0000","last_became_active":"2024-03-11T19:19:42.381750+0000","last_became_peered":"2024-03-11T19:19:42.381750+0000","last_unstale":"2024-03-11T19:21:32.315380+0000","last_undegraded":"2024-03-11T19:21:32.315380+0000","last_fullsized":"2024-03-11T19:21:32.315380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:54:20.851516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313134+0000","last_change":"2024-03-11T19:19:46.801276+0000","last_active":"2024-03-11T19:21:32.313134+0000","last_peered":"2024-03-11T19:21:32.313134+0000","last_clean":"2024-03-11T19:21:32.313134+0000","last_became_active":"2024-03-11T19:19:46.800791+0000","last_became_peered":"2024-03-11T19:19:46.800791+0000","last_unstale":"2024-03-11T19:21:32.313134+0000","last_undegraded":"2024-03-11T19:21:32.313134+0000","last_fullsized":"2024-03-11T19:21:32.313134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:05.418175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416728+0000","last_change":"2024-03-11T19:19:41.392788+0000","last_active":"2024-03-11T19:21:32.416728+0000","last_peered":"2024-03-11T19:21:32.416728+0000","last_clean":"2024-03-11T19:21:32.416728+0000","last_became_active":"2024-03-11T19:19:41.392483+0000","last_became_peered":"2024-03-11T19:19:41.392483+0000","last_unstale":"2024-03-11T19:21:32.416728+0000","last_undegraded":"2024-03-11T19:21:32.416728+0000","last_fullsized":"2024-03-11T19:21:32.416728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:55.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481783+0000","last_change":"2024-03-11T19:19:42.394123+0000","last_active":"2024-03-11T19:21:32.481783+0000","last_peered":"2024-03-11T19:21:32.481783+0000","last_clean":"2024-03-11T19:21:32.481783+0000","last_became_active":"2024-03-11T19:19:42.393785+0000","last_became_peered":"2024-03-11T19:19:42.393785+0000","last_unstale":"2024-03-11T19:21:32.481783+0000","last_undegraded":"2024-03-11T19:21:32.481783+0000","last_fullsized":"2024-03-11T19:21:32.481783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:46:24.299553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416697+0000","last_change":"2024-03-11T19:20:31.328899+0000","last_active":"2024-03-11T19:21:32.416697+0000","last_peered":"2024-03-11T19:21:32.416697+0000","last_clean":"2024-03-11T19:21:32.416697+0000","last_became_active":"2024-03-11T19:20:31.328401+0000","last_became_peered":"2024-03-11T19:20:31.328401+0000","last_unstale":"2024-03-11T19:21:32.416697+0000","last_undegraded":"2024-03-11T19:21:32.416697+0000","last_fullsized":"2024-03-11T19:21:32.416697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:41.749529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480892+0000","last_change":"2024-03-11T19:19:41.394099+0000","last_active":"2024-03-11T19:21:32.480892+0000","last_peered":"2024-03-11T19:21:32.480892+0000","last_clean":"2024-03-11T19:21:32.480892+0000","last_became_active":"2024-03-11T19:19:41.393780+0000","last_became_peered":"2024-03-11T19:19:41.393780+0000","last_unstale":"2024-03-11T19:21:32.480892+0000","last_undegraded":"2024-03-11T19:21:32.480892+0000","last_fullsized":"2024-03-11T19:21:32.480892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:38:16.163212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322159+0000","last_change":"2024-03-11T19:19:42.363963+0000","last_active":"2024-03-11T19:21:32.322159+0000","last_peered":"2024-03-11T19:21:32.322159+0000","last_clean":"2024-03-11T19:21:32.322159+0000","last_became_active":"2024-03-11T19:19:42.363315+0000","last_became_peered":"2024-03-11T19:19:42.363315+0000","last_unstale":"2024-03-11T19:21:32.322159+0000","last_undegraded":"2024-03-11T19:21:32.322159+0000","last_fullsized":"2024-03-11T19:21:32.322159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:13:14.828230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.318201+0000","last_change":"2024-03-11T19:19:46.808879+0000","last_active":"2024-03-11T19:21:32.318201+0000","last_peered":"2024-03-11T19:21:32.318201+0000","last_clean":"2024-03-11T19:21:32.318201+0000","last_became_active":"2024-03-11T19:19:46.808428+0000","last_became_peered":"2024-03-11T19:19:46.808428+0000","last_unstale":"2024-03-11T19:21:32.318201+0000","last_undegraded":"2024-03-11T19:21:32.318201+0000","last_fullsized":"2024-03-11T19:21:32.318201+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:19.024854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314925+0000","last_change":"2024-03-11T19:19:41.395487+0000","last_active":"2024-03-11T19:21:32.314925+0000","last_peered":"2024-03-11T19:21:32.314925+0000","last_clean":"2024-03-11T19:21:32.314925+0000","last_became_active":"2024-03-11T19:19:41.395081+0000","last_became_peered":"2024-03-11T19:19:41.395081+0000","last_unstale":"2024-03-11T19:21:32.314925+0000","last_undegraded":"2024-03-11T19:21:32.314925+0000","last_fullsized":"2024-03-11T19:21:32.314925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:54:58.855745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480941+0000","last_change":"2024-03-11T19:19:42.378860+0000","last_active":"2024-03-11T19:21:32.480941+0000","last_peered":"2024-03-11T19:21:32.480941+0000","last_clean":"2024-03-11T19:21:32.480941+0000","last_became_active":"2024-03-11T19:19:42.378421+0000","last_became_peered":"2024-03-11T19:19:42.378421+0000","last_unstale":"2024-03-11T19:21:32.480941+0000","last_undegraded":"2024-03-11T19:21:32.480941+0000","last_fullsized":"2024-03-11T19:21:32.480941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:59:08.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416822+0000","last_change":"2024-03-11T19:19:46.815770+0000","last_active":"2024-03-11T19:21:32.416822+0000","last_peered":"2024-03-11T19:21:32.416822+0000","last_clean":"2024-03-11T19:21:32.416822+0000","last_became_active":"2024-03-11T19:19:46.814982+0000","last_became_peered":"2024-03-11T19:19:46.814982+0000","last_unstale":"2024-03-11T19:21:32.416822+0000","last_undegraded":"2024-03-11T19:21:32.416822+0000","last_fullsized":"2024-03-11T19:21:32.416822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:19.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313252+0000","last_change":"2024-03-11T19:19:41.397753+0000","last_active":"2024-03-11T19:21:32.313252+0000","last_peered":"2024-03-11T19:21:32.313252+0000","last_clean":"2024-03-11T19:21:32.313252+0000","last_became_active":"2024-03-11T19:19:41.397200+0000","last_became_peered":"2024-03-11T19:19:41.397200+0000","last_unstale":"2024-03-11T19:21:32.313252+0000","last_undegraded":"2024-03-11T19:21:32.313252+0000","last_fullsized":"2024-03-11T19:21:32.313252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:38:38.326990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.314800+0000","last_change":"2024-03-11T19:19:42.387285+0000","last_active":"2024-03-11T19:21:32.314800+0000","last_peered":"2024-03-11T19:21:32.314800+0000","last_clean":"2024-03-11T19:21:32.314800+0000","last_became_active":"2024-03-11T19:19:42.386919+0000","last_became_peered":"2024-03-11T19:19:42.386919+0000","last_unstale":"2024-03-11T19:21:32.314800+0000","last_undegraded":"2024-03-11T19:21:32.314800+0000","last_fullsized":"2024-03-11T19:21:32.314800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.773177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.416483+0000","last_change":"2024-03-11T19:19:46.934177+0000","last_active":"2024-03-11T19:21:32.416483+0000","last_peered":"2024-03-11T19:21:32.416483+0000","last_clean":"2024-03-11T19:21:32.416483+0000","last_became_active":"2024-03-11T19:19:46.933699+0000","last_became_peered":"2024-03-11T19:19:46.933699+0000","last_unstale":"2024-03-11T19:21:32.416483+0000","last_undegraded":"2024-03-11T19:21:32.416483+0000","last_fullsized":"2024-03-11T19:21:32.416483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:09:17.433172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.322216+0000","last_change":"2024-03-11T19:19:41.396888+0000","last_active":"2024-03-11T19:21:32.322216+0000","last_peered":"2024-03-11T19:21:32.322216+0000","last_clean":"2024-03-11T19:21:32.322216+0000","last_became_active":"2024-03-11T19:19:41.396640+0000","last_became_peered":"2024-03-11T19:19:41.396640+0000","last_unstale":"2024-03-11T19:21:32.322216+0000","last_undegraded":"2024-03-11T19:21:32.322216+0000","last_fullsized":"2024-03-11T19:21:32.322216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:00.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481382+0000","last_change":"2024-03-11T19:19:42.370020+0000","last_active":"2024-03-11T19:21:32.481382+0000","last_peered":"2024-03-11T19:21:32.481382+0000","last_clean":"2024-03-11T19:21:32.481382+0000","last_became_active":"2024-03-11T19:19:42.369355+0000","last_became_peered":"2024-03-11T19:19:42.369355+0000","last_unstale":"2024-03-11T19:21:32.481382+0000","last_undegraded":"2024-03-11T19:21:32.481382+0000","last_fullsized":"2024-03-11T19:21:32.481382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:07:13.572419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.480750+0000","last_change":"2024-03-11T19:19:46.818424+0000","last_active":"2024-03-11T19:21:32.480750+0000","last_peered":"2024-03-11T19:21:32.480750+0000","last_clean":"2024-03-11T19:21:32.480750+0000","last_became_active":"2024-03-11T19:19:46.817934+0000","last_became_peered":"2024-03-11T19:19:46.817934+0000","last_unstale":"2024-03-11T19:21:32.480750+0000","last_undegraded":"2024-03-11T19:21:32.480750+0000","last_fullsized":"2024-03-11T19:21:32.480750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:52:56.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313627+0000","last_change":"2024-03-11T19:19:41.399466+0000","last_active":"2024-03-11T19:21:32.313627+0000","last_peered":"2024-03-11T19:21:32.313627+0000","last_clean":"2024-03-11T19:21:32.313627+0000","last_became_active":"2024-03-11T19:19:41.399118+0000","last_became_peered":"2024-03-11T19:19:41.399118+0000","last_unstale":"2024-03-11T19:21:32.313627+0000","last_undegraded":"2024-03-11T19:21:32.313627+0000","last_fullsized":"2024-03-11T19:21:32.313627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:12:46.100596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481777+0000","last_change":"2024-03-11T19:19:42.365437+0000","last_active":"2024-03-11T19:21:32.481777+0000","last_peered":"2024-03-11T19:21:32.481777+0000","last_clean":"2024-03-11T19:21:32.481777+0000","last_became_active":"2024-03-11T19:19:42.365084+0000","last_became_peered":"2024-03-11T19:19:42.365084+0000","last_unstale":"2024-03-11T19:21:32.481777+0000","last_undegraded":"2024-03-11T19:21:32.481777+0000","last_fullsized":"2024-03-11T19:21:32.481777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:45:35.239100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481460+0000","last_change":"2024-03-11T19:19:46.825067+0000","last_active":"2024-03-11T19:21:32.481460+0000","last_peered":"2024-03-11T19:21:32.481460+0000","last_clean":"2024-03-11T19:21:32.481460+0000","last_became_active":"2024-03-11T19:19:46.821438+0000","last_became_peered":"2024-03-11T19:19:46.821438+0000","last_unstale":"2024-03-11T19:21:32.481460+0000","last_undegraded":"2024-03-11T19:21:32.481460+0000","last_fullsized":"2024-03-11T19:21:32.481460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:50:08.828669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.357475+0000","last_change":"2024-03-11T19:19:41.403788+0000","last_active":"2024-03-11T19:21:32.357475+0000","last_peered":"2024-03-11T19:21:32.357475+0000","last_clean":"2024-03-11T19:21:32.357475+0000","last_became_active":"2024-03-11T19:19:41.403479+0000","last_became_peered":"2024-03-11T19:19:41.403479+0000","last_unstale":"2024-03-11T19:21:32.357475+0000","last_undegraded":"2024-03-11T19:21:32.357475+0000","last_fullsized":"2024-03-11T19:21:32.357475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:58:03.407100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481026+0000","last_change":"2024-03-11T19:19:42.381891+0000","last_active":"2024-03-11T19:21:32.481026+0000","last_peered":"2024-03-11T19:21:32.481026+0000","last_clean":"2024-03-11T19:21:32.481026+0000","last_became_active":"2024-03-11T19:19:42.381573+0000","last_became_peered":"2024-03-11T19:19:42.381573+0000","last_unstale":"2024-03-11T19:21:32.481026+0000","last_undegraded":"2024-03-11T19:21:32.481026+0000","last_fullsized":"2024-03-11T19:21:32.481026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:25.967035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.321995+0000","last_change":"2024-03-11T19:20:31.361853+0000","last_active":"2024-03-11T19:21:32.321995+0000","last_peered":"2024-03-11T19:21:32.321995+0000","last_clean":"2024-03-11T19:21:32.321995+0000","last_became_active":"2024-03-11T19:20:31.361381+0000","last_became_peered":"2024-03-11T19:20:31.361381+0000","last_unstale":"2024-03-11T19:21:32.321995+0000","last_undegraded":"2024-03-11T19:21:32.321995+0000","last_fullsized":"2024-03-11T19:21:32.321995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:45.765582+0000","last_clean_scrub_stamp":"2024-03-11T19:19:45.765582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:56:49.225542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.414793+0000","last_change":"2024-03-11T19:19:41.394613+0000","last_active":"2024-03-11T19:21:32.414793+0000","last_peered":"2024-03-11T19:21:32.414793+0000","last_clean":"2024-03-11T19:21:32.414793+0000","last_became_active":"2024-03-11T19:19:41.394252+0000","last_became_peered":"2024-03-11T19:19:41.394252+0000","last_unstale":"2024-03-11T19:21:32.414793+0000","last_undegraded":"2024-03-11T19:21:32.414793+0000","last_fullsized":"2024-03-11T19:21:32.414793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:40.366833+0000","last_clean_scrub_stamp":"2024-03-11T19:19:40.366833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:05:50.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.482094+0000","last_change":"2024-03-11T19:19:42.366706+0000","last_active":"2024-03-11T19:21:32.482094+0000","last_peered":"2024-03-11T19:21:32.482094+0000","last_clean":"2024-03-11T19:21:32.482094+0000","last_became_active":"2024-03-11T19:19:42.366389+0000","last_became_peered":"2024-03-11T19:19:42.366389+0000","last_unstale":"2024-03-11T19:21:32.482094+0000","last_undegraded":"2024-03-11T19:21:32.482094+0000","last_fullsized":"2024-03-11T19:21:32.482094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:15:08.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481070+0000","last_change":"2024-03-11T19:19:42.367886+0000","last_active":"2024-03-11T19:21:32.481070+0000","last_peered":"2024-03-11T19:21:32.481070+0000","last_clean":"2024-03-11T19:21:32.481070+0000","last_became_active":"2024-03-11T19:19:42.367543+0000","last_became_peered":"2024-03-11T19:19:42.367543+0000","last_unstale":"2024-03-11T19:21:32.481070+0000","last_undegraded":"2024-03-11T19:21:32.481070+0000","last_fullsized":"2024-03-11T19:21:32.481070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:15:01.606205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313549+0000","last_change":"2024-03-11T19:19:42.370189+0000","last_active":"2024-03-11T19:21:32.313549+0000","last_peered":"2024-03-11T19:21:32.313549+0000","last_clean":"2024-03-11T19:21:32.313549+0000","last_became_active":"2024-03-11T19:19:42.369553+0000","last_became_peered":"2024-03-11T19:19:42.369553+0000","last_unstale":"2024-03-11T19:21:32.313549+0000","last_undegraded":"2024-03-11T19:21:32.313549+0000","last_fullsized":"2024-03-11T19:21:32.313549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:10:09.973215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313241+0000","last_change":"2024-03-11T19:19:42.380853+0000","last_active":"2024-03-11T19:21:32.313241+0000","last_peered":"2024-03-11T19:21:32.313241+0000","last_clean":"2024-03-11T19:21:32.313241+0000","last_became_active":"2024-03-11T19:19:42.380502+0000","last_became_peered":"2024-03-11T19:19:42.380502+0000","last_unstale":"2024-03-11T19:21:32.313241+0000","last_undegraded":"2024-03-11T19:21:32.313241+0000","last_fullsized":"2024-03-11T19:21:32.313241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:22:20.832801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481764+0000","last_change":"2024-03-11T19:19:42.382850+0000","last_active":"2024-03-11T19:21:32.481764+0000","last_peered":"2024-03-11T19:21:32.481764+0000","last_clean":"2024-03-11T19:21:32.481764+0000","last_became_active":"2024-03-11T19:19:42.382134+0000","last_became_peered":"2024-03-11T19:19:42.382134+0000","last_unstale":"2024-03-11T19:21:32.481764+0000","last_undegraded":"2024-03-11T19:21:32.481764+0000","last_fullsized":"2024-03-11T19:21:32.481764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:49.859290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.413668+0000","last_change":"2024-03-11T19:19:42.386882+0000","last_active":"2024-03-11T19:21:32.413668+0000","last_peered":"2024-03-11T19:21:32.413668+0000","last_clean":"2024-03-11T19:21:32.413668+0000","last_became_active":"2024-03-11T19:19:42.386280+0000","last_became_peered":"2024-03-11T19:19:42.386280+0000","last_unstale":"2024-03-11T19:21:32.413668+0000","last_undegraded":"2024-03-11T19:21:32.413668+0000","last_fullsized":"2024-03-11T19:21:32.413668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:30:10.891093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481338+0000","last_change":"2024-03-11T19:19:42.368025+0000","last_active":"2024-03-11T19:21:32.481338+0000","last_peered":"2024-03-11T19:21:32.481338+0000","last_clean":"2024-03-11T19:21:32.481338+0000","last_became_active":"2024-03-11T19:19:42.367695+0000","last_became_peered":"2024-03-11T19:19:42.367695+0000","last_unstale":"2024-03-11T19:21:32.481338+0000","last_undegraded":"2024-03-11T19:21:32.481338+0000","last_fullsized":"2024-03-11T19:21:32.481338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:11:44.776358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.313994+0000","last_change":"2024-03-11T19:19:42.364940+0000","last_active":"2024-03-11T19:21:32.313994+0000","last_peered":"2024-03-11T19:21:32.313994+0000","last_clean":"2024-03-11T19:21:32.313994+0000","last_became_active":"2024-03-11T19:19:42.364620+0000","last_became_peered":"2024-03-11T19:19:42.364620+0000","last_unstale":"2024-03-11T19:21:32.313994+0000","last_undegraded":"2024-03-11T19:21:32.313994+0000","last_fullsized":"2024-03-11T19:21:32.313994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:05:11.415716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315591+0000","last_change":"2024-03-11T19:19:42.392468+0000","last_active":"2024-03-11T19:21:32.315591+0000","last_peered":"2024-03-11T19:21:32.315591+0000","last_clean":"2024-03-11T19:21:32.315591+0000","last_became_active":"2024-03-11T19:19:42.390521+0000","last_became_peered":"2024-03-11T19:19:42.390521+0000","last_unstale":"2024-03-11T19:21:32.315591+0000","last_undegraded":"2024-03-11T19:21:32.315591+0000","last_fullsized":"2024-03-11T19:21:32.315591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:51:21.487796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.356884+0000","last_change":"2024-03-11T19:19:42.393130+0000","last_active":"2024-03-11T19:21:32.356884+0000","last_peered":"2024-03-11T19:21:32.356884+0000","last_clean":"2024-03-11T19:21:32.356884+0000","last_became_active":"2024-03-11T19:19:42.392845+0000","last_became_peered":"2024-03-11T19:19:42.392845+0000","last_unstale":"2024-03-11T19:21:32.356884+0000","last_undegraded":"2024-03-11T19:21:32.356884+0000","last_fullsized":"2024-03-11T19:21:32.356884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:17:27.278574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.315268+0000","last_change":"2024-03-11T19:19:42.385890+0000","last_active":"2024-03-11T19:21:32.315268+0000","last_peered":"2024-03-11T19:21:32.315268+0000","last_clean":"2024-03-11T19:21:32.315268+0000","last_became_active":"2024-03-11T19:19:42.381939+0000","last_became_peered":"2024-03-11T19:19:42.381939+0000","last_unstale":"2024-03-11T19:21:32.315268+0000","last_undegraded":"2024-03-11T19:21:32.315268+0000","last_fullsized":"2024-03-11T19:21:32.315268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:47:11.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481168+0000","last_change":"2024-03-11T19:19:42.392856+0000","last_active":"2024-03-11T19:21:32.481168+0000","last_peered":"2024-03-11T19:21:32.481168+0000","last_clean":"2024-03-11T19:21:32.481168+0000","last_became_active":"2024-03-11T19:19:42.392554+0000","last_became_peered":"2024-03-11T19:19:42.392554+0000","last_unstale":"2024-03-11T19:21:32.481168+0000","last_undegraded":"2024-03-11T19:21:32.481168+0000","last_fullsized":"2024-03-11T19:21:32.481168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:54:04.840145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.412859+0000","last_change":"2024-03-11T19:19:42.383082+0000","last_active":"2024-03-11T19:21:32.412859+0000","last_peered":"2024-03-11T19:21:32.412859+0000","last_clean":"2024-03-11T19:21:32.412859+0000","last_became_active":"2024-03-11T19:19:42.381546+0000","last_became_peered":"2024-03-11T19:19:42.381546+0000","last_unstale":"2024-03-11T19:21:32.412859+0000","last_undegraded":"2024-03-11T19:21:32.412859+0000","last_fullsized":"2024-03-11T19:21:32.412859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:20.493148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:21:32.481038+0000","last_change":"2024-03-11T19:19:42.369136+0000","last_active":"2024-03-11T19:21:32.481038+0000","last_peered":"2024-03-11T19:21:32.481038+0000","last_clean":"2024-03-11T19:21:32.481038+0000","last_became_active":"2024-03-11T19:19:42.368807+0000","last_became_peered":"2024-03-11T19:19:42.368807+0000","last_unstale":"2024-03-11T19:21:32.481038+0000","last_undegraded":"2024-03-11T19:21:32.481038+0000","last_fullsized":"2024-03-11T19:21:32.481038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:19:41.371408+0000","last_clean_scrub_stamp":"2024-03-11T19:19:41.371408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:16:34.288622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281180,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66724,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":633,"num_read_kb":389,"num_write":170,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":200172,"data_compressed":4818,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":523154,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":1177610,"data_compressed":64022,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":523368,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":523266,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299457,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1119338,"data_compressed":62416,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":518544,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":581036,"data_compressed":28258,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":519932,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":582386,"data_compressed":28258,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":518738,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888147,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":523726,"data_compressed":26652,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1114394,"data_compressed":62416,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34120,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35764,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62582,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63932,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64692,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:22:00.740 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-11T19:22:00.740 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-11T19:22:00.740 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-11T19:22:00.740 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-11T19:22:01.230 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:01.620 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:01 smithi072 ceph-mon[31695]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:22:01.620 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:01 smithi072 ceph-mon[31695]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:01 smithi106 ceph-mon[35889]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:22:01.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:01 smithi106 ceph-mon[35889]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:01.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:01 smithi143 ceph-mon[36013]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:22:01.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:01 smithi143 ceph-mon[36013]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:03.438 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:03 smithi072 ceph-mon[31695]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:03.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:03 smithi106 ceph-mon[35889]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:03.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:03 smithi143 ceph-mon[36013]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:03.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.733+0000 7f254f000700 1 -- 172.21.15.72:0/3982401987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480740a0 msgr2=0x7f2548074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.733+0000 7f254f000700 1 --2- 172.21.15.72:0/3982401987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480740a0 0x7f2548074480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f253c009990 tx=0x7f253c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 -- 172.21.15.72:0/3982401987 shutdown_connections 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 --2- 172.21.15.72:0/3982401987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f254810e3a0 0x7f2548110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 --2- 172.21.15.72:0/3982401987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480740a0 0x7f2548074480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 --2- 172.21.15.72:0/3982401987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25480749c0 0x7f254810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 -- 172.21.15.72:0/3982401987 >> 172.21.15.72:0/3982401987 conn(0x7f25480fd670 msgr2=0x7f25480ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 -- 172.21.15.72:0/3982401987 shutdown_connections 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.734+0000 7f254f000700 1 -- 172.21.15.72:0/3982401987 wait complete. 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.735+0000 7f254f000700 1 Processor -- start 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 -- start start 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 0x7f254819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:03.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f254810e3a0 0x7f254819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254819f280 con 0x7f254810e3a0 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254819f3f0 con 0x7f25480740a0 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254f000700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254819f5b0 con 0x7f25480749c0 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254dc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254d200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 0x7f254819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254dc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36718/0 (socket says 172.21.15.72:36718) 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.736+0000 7f254dc00700 1 -- 172.21.15.72:0/2594242445 learned_addr learned my addr 172.21.15.72:0/2594242445 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:03.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254e600700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f254810e3a0 0x7f254819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 -- 172.21.15.72:0/2594242445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 msgr2=0x7f254819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 -- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f254810e3a0 msgr2=0x7f254819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f254810e3a0 0x7f254819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 -- 172.21.15.72:0/2594242445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534009520 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254dc00700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f254d200700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 0x7f254819a930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f253400e9f0 tx=0x7f253400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.737+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400cb70 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.738+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f253400ee90 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.738+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c0095f0 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.738+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534003d80 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.738+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25481a6f20 con 0x7f25480749c0 2024-03-11T19:22:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.739+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25481095b0 con 0x7f25480749c0 2024-03-11T19:22:03.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.741+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2534003ee0 con 0x7f25480749c0 2024-03-11T19:22:03.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.742+0000 7f2546000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 0x7f25300a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:03.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.742+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f2534014070 con 0x7f25480749c0 2024-03-11T19:22:03.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.742+0000 7f254dc00700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 0x7f25300a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:03.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.743+0000 7f254dc00700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 0x7f25300a4000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f253c006010 tx=0x7f253c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:03.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:03.745+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2534096860 con 0x7f25480749c0 2024-03-11T19:22:04.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.011+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f254819b700 con 0x7f25480749c0 2024-03-11T19:22:04.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.012+0000 7f2546000700 1 -- 172.21.15.72:0/2594242445 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2534017e90 con 0x7f25480749c0 2024-03-11T19:22:04.016 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:04.017 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-11T19:22:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 msgr2=0x7f25300a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 0x7f25300a4000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f253c006010 tx=0x7f253c00b260 comp rx=0 tx=0).stop 2024-03-11T19:22:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 msgr2=0x7f254819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 0x7f254819a930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f253400e9f0 tx=0x7f253400ed00 comp rx=0 tx=0).stop 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 shutdown_connections 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f25300a1b40 0x7f25300a4000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f254810e3a0 0x7f254819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25480740a0 0x7f254819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 --2- 172.21.15.72:0/2594242445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25480749c0 0x7f254819a930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.015+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 >> 172.21.15.72:0/2594242445 conn(0x7f25480fd670 msgr2=0x7f25480fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.016+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 shutdown_connections 2024-03-11T19:22:04.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:04.016+0000 7f254f000700 1 -- 172.21.15.72:0/2594242445 wait complete. 2024-03-11T19:22:04.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2594242445' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:22:04.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2594242445' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:22:04.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2594242445' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:22:05.111 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-11T19:22:05.112 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 -- 172.21.15.72:0/3016730138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 msgr2=0x7f435c113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 --2- 172.21.15.72:0/3016730138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c113e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4354009be0 tx=0x7f435402bef0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 -- 172.21.15.72:0/3016730138 shutdown_connections 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 --2- 172.21.15.72:0/3016730138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c113e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 --2- 172.21.15.72:0/3016730138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 0x7f435c1118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 --2- 172.21.15.72:0/3016730138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 0x7f435c10a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 -- 172.21.15.72:0/3016730138 >> 172.21.15.72:0/3016730138 conn(0x7f435c06e390 msgr2=0x7f435c0707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 -- 172.21.15.72:0/3016730138 shutdown_connections 2024-03-11T19:22:05.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.330+0000 7f4361a00640 1 -- 172.21.15.72:0/3016730138 wait complete. 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.331+0000 7f4361a00640 1 Processor -- start 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.331+0000 7f4361a00640 1 -- start start 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.331+0000 7f4361a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 0x7f435c1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4361a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 0x7f435c1a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4361a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4361a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a96a0 con 0x7f435c10af70 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4361a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a9810 con 0x7f435c108180 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4361a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a99d0 con 0x7f435c10aaf0 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 0x7f435c1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51018/0 (socket says 172.21.15.72:51018) 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 -- 172.21.15.72:0/2932013372 learned_addr learned my addr 172.21.15.72:0/2932013372 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f4353e00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 0x7f435c1a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 -- 172.21.15.72:0/2932013372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 msgr2=0x7f435c1a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 0x7f435c1a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 -- 172.21.15.72:0/2932013372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 msgr2=0x7f435c1a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 0x7f435c1a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.332+0000 7f435b400640 1 -- 172.21.15.72:0/2932013372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009560 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f435b400640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4354004af0 tx=0x7f4354033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354003d80 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f435402ce40 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4354009840 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354034960 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.333+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c1a9f70 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.334+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c112fd0 con 0x7f435c10af70 2024-03-11T19:22:05.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.336+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4354034ac0 con 0x7f435c10af70 2024-03-11T19:22:05.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.338+0000 7f4353400640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 0x7f43400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.338+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f43540efe90 con 0x7f435c10af70 2024-03-11T19:22:05.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.338+0000 7f435aa00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 0x7f43400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.339+0000 7f435aa00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 0x7f43400a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f434c009530 tx=0x7f434c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:05.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.343+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43540b8d70 con 0x7f435c10af70 2024-03-11T19:22:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:05 smithi072 ceph-mon[31695]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:05.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.545+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f435c02cea0 con 0x7f435c10af70 2024-03-11T19:22:05.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.548+0000 7f4353400640 1 -- 172.21.15.72:0/2932013372 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7f43540bc430 con 0x7f435c10af70 2024-03-11T19:22:05.552 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:05.553 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 msgr2=0x7f43400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 0x7f43400a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f434c009530 tx=0x7f434c009260 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 msgr2=0x7f435c1a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4354004af0 tx=0x7f4354033d00 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 shutdown_connections 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f43400a1a30 0x7f43400a3ef0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f435c10af70 0x7f435c1a9160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f435c10aaf0 0x7f435c1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 --2- 172.21.15.72:0/2932013372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f435c108180 0x7f435c1a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 >> 172.21.15.72:0/2932013372 conn(0x7f435c06e390 msgr2=0x7f435c06ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 shutdown_connections 2024-03-11T19:22:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.551+0000 7f4361a00640 1 -- 172.21.15.72:0/2932013372 wait complete. 2024-03-11T19:22:05.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:22:05.575 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:22:05.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:05 smithi106 ceph-mon[35889]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:05.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:05 smithi143 ceph-mon[36013]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.788+0000 7fba86200640 1 -- 172.21.15.72:0/1228288316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801045b0 msgr2=0x7fba80104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.788+0000 7fba86200640 1 --2- 172.21.15.72:0/1228288316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801045b0 0x7fba80104a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba68009900 tx=0x7fba6802bc40 comp rx=0 tx=0).stop 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 -- 172.21.15.72:0/1228288316 shutdown_connections 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 --2- 172.21.15.72:0/1228288316 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba8010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 --2- 172.21.15.72:0/1228288316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801045b0 0x7fba80104a30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 --2- 172.21.15.72:0/1228288316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801033c0 0x7fba801037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 -- 172.21.15.72:0/1228288316 >> 172.21.15.72:0/1228288316 conn(0x7fba800fe930 msgr2=0x7fba80100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 -- 172.21.15.72:0/1228288316 shutdown_connections 2024-03-11T19:22:05.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.789+0000 7fba86200640 1 -- 172.21.15.72:0/1228288316 wait complete. 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 Processor -- start 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 -- start start 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801045b0 0x7fba80199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba801a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a0f00 con 0x7fba801033c0 2024-03-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a1070 con 0x7fba80104f70 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba86200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801a1230 con 0x7fba801045b0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba7f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba7f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51040/0 (socket says 172.21.15.72:51040) 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba7f400640 1 -- 172.21.15.72:0/3205675157 learned_addr learned my addr 172.21.15.72:0/3205675157 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba77e00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801045b0 0x7fba80199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.790+0000 7fba7fe00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba801a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 -- 172.21.15.72:0/3205675157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801045b0 msgr2=0x7fba80199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801045b0 0x7fba80199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 -- 172.21.15.72:0/3205675157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 msgr2=0x7fba801a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba801a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 -- 172.21.15.72:0/3205675157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70009490 con 0x7fba801033c0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7fe00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba801a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7f400640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000cde0 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba70010910 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba68009560 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70018a10 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.791+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801a18b0 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.793+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8010a760 con 0x7fba801033c0 2024-03-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.794+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fba70010a80 con 0x7fba801033c0 2024-03-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.796+0000 7fba7cc00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 0x7fba640a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.796+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fba70014070 con 0x7fba801033c0 2024-03-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.796+0000 7fba77e00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 0x7fba640a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.797+0000 7fba77e00640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 0x7fba640a3f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba68009230 tx=0x7fba68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:05.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.800+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba70096cf0 con 0x7fba801033c0 2024-03-11T19:22:06.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.996+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba8002ce40 con 0x7fba801033c0 2024-03-11T19:22:06.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.997+0000 7fba7cc00640 1 -- 172.21.15.72:0/3205675157 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba7009a3b0 con 0x7fba801033c0 2024-03-11T19:22:06.001 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:22:06.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 msgr2=0x7fba640a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 0x7fba640a3f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba68009230 tx=0x7fba68005fd0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 msgr2=0x7fba80199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 shutdown_connections 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fba640a1ab0 0x7fba640a3f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba80104f70 0x7fba801a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba801045b0 0x7fba80199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 --2- 172.21.15.72:0/3205675157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fba801033c0 0x7fba80199410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:05.999+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 >> 172.21.15.72:0/3205675157 conn(0x7fba800fe930 msgr2=0x7fba80107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:06.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.000+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 shutdown_connections 2024-03-11T19:22:06.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.000+0000 7fba86200640 1 -- 172.21.15.72:0/3205675157 wait complete. 2024-03-11T19:22:06.024 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:22:06.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 -- 172.21.15.72:0/2610414157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0988104f70 msgr2=0x7f098810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 --2- 172.21.15.72:0/2610414157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0988104f70 0x7f098810b800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0980009ca0 tx=0x7f098002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 -- 172.21.15.72:0/2610414157 shutdown_connections 2024-03-11T19:22:06.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 --2- 172.21.15.72:0/2610414157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0988104f70 0x7f098810b800 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 --2- 172.21.15.72:0/2610414157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09881045b0 0x7f0988104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 --2- 172.21.15.72:0/2610414157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881033c0 0x7f09881037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 -- 172.21.15.72:0/2610414157 >> 172.21.15.72:0/2610414157 conn(0x7f09880fe930 msgr2=0x7f0988100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:06.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.234+0000 7f098cc00640 1 -- 172.21.15.72:0/2610414157 shutdown_connections 2024-03-11T19:22:06.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.235+0000 7f098cc00640 1 -- 172.21.15.72:0/2610414157 wait complete. 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 Processor -- start 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 -- start start 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 0x7f09881993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 0x7f0988199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0988104f70 0x7f09881a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881a0eb0 con 0x7f09881033c0 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881a1020 con 0x7f09881045b0 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.236+0000 7f098cc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881a11e0 con 0x7f0988104f70 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 0x7f09881993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 0x7f0988199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 0x7f09881993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51068/0 (socket says 172.21.15.72:51068) 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985e00640 1 -- 172.21.15.72:0/3401585199 learned_addr learned my addr 172.21.15.72:0/3401585199 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0986800640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0988104f70 0x7f09881a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 -- 172.21.15.72:0/3401585199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0988104f70 msgr2=0x7f09881a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0988104f70 0x7f09881a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 -- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 msgr2=0x7f09881993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 0x7f09881993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 -- 172.21.15.72:0/3401585199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980009900 con 0x7f09881045b0 2024-03-11T19:22:06.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.237+0000 7f0985400640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 0x7f0988199900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f097400e9f0 tx=0x7f097400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:06.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.238+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400ccc0 con 0x7f09881045b0 2024-03-11T19:22:06.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.238+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f097400ce20 con 0x7f09881045b0 2024-03-11T19:22:06.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.238+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09881a14c0 con 0x7f09881045b0 2024-03-11T19:22:06.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.238+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974018920 con 0x7f09881045b0 2024-03-11T19:22:06.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.238+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09881a19d0 con 0x7f09881045b0 2024-03-11T19:22:06.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.239+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098810a680 con 0x7f09881045b0 2024-03-11T19:22:06.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.242+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0974018a80 con 0x7f09881045b0 2024-03-11T19:22:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.243+0000 7f097ea00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 0x7f096c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.243+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f0974014070 con 0x7f09881045b0 2024-03-11T19:22:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.243+0000 7f0985e00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 0x7f096c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.244+0000 7f0985e00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 0x7f096c0a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09780058c0 tx=0x7f0978005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.245+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0974096940 con 0x7f09881045b0 2024-03-11T19:22:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2932013372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:06.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3205675157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:06.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.446+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f098810ef30 con 0x7f09881045b0 2024-03-11T19:22:06.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.447+0000 7f097ea00640 1 -- 172.21.15.72:0/3401585199 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f097409a000 con 0x7f09881045b0 2024-03-11T19:22:06.451 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:06.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 msgr2=0x7f096c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 0x7f096c0a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09780058c0 tx=0x7f0978005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 msgr2=0x7f0988199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 0x7f0988199900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f097400e9f0 tx=0x7f097400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 shutdown_connections 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f096c0a1a30 0x7f096c0a3ef0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0988104f70 0x7f09881a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09881045b0 0x7f0988199900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 --2- 172.21.15.72:0/3401585199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f09881033c0 0x7f09881993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 >> 172.21.15.72:0/3401585199 conn(0x7f09880fe930 msgr2=0x7f0988107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 shutdown_connections 2024-03-11T19:22:06.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.450+0000 7f098cc00640 1 -- 172.21.15.72:0/3401585199 wait complete. 2024-03-11T19:22:06.473 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:22:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2932013372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:06.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3205675157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 -- 172.21.15.72:0/3962477535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec1033c0 msgr2=0x7f31ec1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 --2- 172.21.15.72:0/3962477535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec1033c0 0x7f31ec1037e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009900 tx=0x7f31dc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 -- 172.21.15.72:0/3962477535 shutdown_connections 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 --2- 172.21.15.72:0/3962477535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec104f70 0x7f31ec10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 --2- 172.21.15.72:0/3962477535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 --2- 172.21.15.72:0/3962477535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec1033c0 0x7f31ec1037e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 -- 172.21.15.72:0/3962477535 >> 172.21.15.72:0/3962477535 conn(0x7f31ec0fe930 msgr2=0x7f31ec100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 -- 172.21.15.72:0/3962477535 shutdown_connections 2024-03-11T19:22:06.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.669+0000 7f31f3a00640 1 -- 172.21.15.72:0/3962477535 wait complete. 2024-03-11T19:22:06.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.670+0000 7f31f3a00640 1 Processor -- start 2024-03-11T19:22:06.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.670+0000 7f31f3a00640 1 -- start start 2024-03-11T19:22:06.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec104f70 0x7f31ec1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1a0f80 con 0x7f31ec1033c0 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1a10f0 con 0x7f31ec1045b0 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1a12b0 con 0x7f31ec104f70 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f1000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f1000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51084/0 (socket says 172.21.15.72:51084) 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f1000640 1 -- 172.21.15.72:0/2700332034 learned_addr learned my addr 172.21.15.72:0/2700332034 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f1a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec104f70 0x7f31ec1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 -- 172.21.15.72:0/2700332034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec104f70 msgr2=0x7f31ec1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec104f70 0x7f31ec1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 -- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 msgr2=0x7f31ec199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31e3e00640 1 -- 172.21.15.72:0/2700332034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4009840 con 0x7f31ec1045b0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.671+0000 7f31f1000640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31e3e00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec1999d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31d800e9f0 tx=0x7f31d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8009620 con 0x7f31ec1045b0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc009560 con 0x7f31ec1045b0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31d800ccc0 con 0x7f31ec1045b0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8004a00 con 0x7f31ec1045b0 2024-03-11T19:22:06.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.672+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec1a1910 con 0x7f31ec1045b0 2024-03-11T19:22:06.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.673+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec10a7b0 con 0x7f31ec1045b0 2024-03-11T19:22:06.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.676+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f31d8004b60 con 0x7f31ec1045b0 2024-03-11T19:22:06.681 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.677+0000 7f31ea000640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 0x7f31d00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:06.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.678+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f31d8014070 con 0x7f31ec1045b0 2024-03-11T19:22:06.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.678+0000 7f31f1000640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 0x7f31d00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:06.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.678+0000 7f31f1000640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 0x7f31d00a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02c840 tx=0x7f31dc02cd10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:06.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.679+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31d80966b0 con 0x7f31ec1045b0 2024-03-11T19:22:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2932013372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3205675157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:06.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.874+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31ec02cda0 con 0x7f31ec1045b0 2024-03-11T19:22:06.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.875+0000 7f31ea000640 1 -- 172.21.15.72:0/2700332034 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31d8099d70 con 0x7f31ec1045b0 2024-03-11T19:22:06.879 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:22:06.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 msgr2=0x7f31d00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 0x7f31d00a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02c840 tx=0x7f31dc02cd10 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 msgr2=0x7f31ec1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec1999d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31d800e9f0 tx=0x7f31d800edf0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 shutdown_connections 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f31d00a1a30 0x7f31d00a3ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec104f70 0x7f31ec1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31ec1045b0 0x7f31ec1999d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 --2- 172.21.15.72:0/2700332034 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31ec1033c0 0x7f31ec199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 >> 172.21.15.72:0/2700332034 conn(0x7f31ec0fe930 msgr2=0x7f31ec107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 shutdown_connections 2024-03-11T19:22:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:06.878+0000 7f31f3a00640 1 -- 172.21.15.72:0/2700332034 wait complete. 2024-03-11T19:22:06.901 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:22:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 -- 172.21.15.72:0/3192100536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 msgr2=0x7f4158108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 --2- 172.21.15.72:0/3192100536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f4158108e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4148009900 tx=0x7f414802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 -- 172.21.15.72:0/3192100536 shutdown_connections 2024-03-11T19:22:07.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 --2- 172.21.15.72:0/3192100536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 0x7f415810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 --2- 172.21.15.72:0/3192100536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f4158108e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 --2- 172.21.15.72:0/3192100536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f4158107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 -- 172.21.15.72:0/3192100536 >> 172.21.15.72:0/3192100536 conn(0x7f4158076740 msgr2=0x7f4158078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 -- 172.21.15.72:0/3192100536 shutdown_connections 2024-03-11T19:22:07.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.107+0000 7f4160800640 1 -- 172.21.15.72:0/3192100536 wait complete. 2024-03-11T19:22:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.108+0000 7f4160800640 1 Processor -- start 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.108+0000 7f4160800640 1 -- start start 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.108+0000 7f4160800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.108+0000 7f4160800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f415819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f4160800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 0x7f41581a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f4160800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581a5330 con 0x7f41581077a0 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f4160800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581a54a0 con 0x7f4158108990 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f4160800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581a5660 con 0x7f4158109350 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51098/0 (socket says 172.21.15.72:51098) 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 -- 172.21.15.72:0/2957630371 learned_addr learned my addr 172.21.15.72:0/2957630371 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415d400640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f415819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415e800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 0x7f41581a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 -- 172.21.15.72:0/2957630371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 msgr2=0x7f41581a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 0x7f41581a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 -- 172.21.15.72:0/2957630371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 msgr2=0x7f415819dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f415819dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415de00640 1 -- 172.21.15.72:0/2957630371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4148009560 con 0x7f41581077a0 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.109+0000 7f415d400640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f415819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:07.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f415de00640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f414c00d950 tx=0x7f414c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c009c30 con 0x7f41581077a0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41581a5940 con 0x7f41581077a0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f414c009d90 con 0x7f41581077a0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00f790 con 0x7f41581077a0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.110+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41581a5e70 con 0x7f41581077a0 2024-03-11T19:22:07.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.111+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41580683c0 con 0x7f41581077a0 2024-03-11T19:22:07.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.115+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f414c010ad0 con 0x7f41581077a0 2024-03-11T19:22:07.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.116+0000 7f4152a00640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 0x7f41400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.116+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f414c0cdd50 con 0x7f41581077a0 2024-03-11T19:22:07.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.117+0000 7f415d400640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 0x7f41400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.117+0000 7f415d400640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 0x7f41400a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4148009230 tx=0x7f4148005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:07.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.119+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f414c096b80 con 0x7f41581077a0 2024-03-11T19:22:07.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.318+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f41581133d0 con 0x7f41581077a0 2024-03-11T19:22:07.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.319+0000 7f4152a00640 1 -- 172.21.15.72:0/2957630371 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f414c017020 con 0x7f41581077a0 2024-03-11T19:22:07.324 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:07.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 msgr2=0x7f41400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 0x7f41400a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4148009230 tx=0x7f4148005fd0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 msgr2=0x7f415819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f414c00d950 tx=0x7f414c00de10 comp rx=0 tx=0).stop 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 shutdown_connections 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f41400a1a30 0x7f41400a3ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4158109350 0x7f41581a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158108990 0x7f415819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 --2- 172.21.15.72:0/2957630371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41581077a0 0x7f415819d840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 >> 172.21.15.72:0/2957630371 conn(0x7f4158076740 msgr2=0x7f415810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 shutdown_connections 2024-03-11T19:22:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.322+0000 7f4160800640 1 -- 172.21.15.72:0/2957630371 wait complete. 2024-03-11T19:22:07.346 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:22:07.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 -- 172.21.15.72:0/2548330917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01045b0 msgr2=0x7fd0d0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2548330917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01045b0 0x7fd0d0104a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009860 tx=0x7fd0bc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 -- 172.21.15.72:0/2548330917 shutdown_connections 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2548330917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d0104f70 0x7fd0d010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2548330917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01045b0 0x7fd0d0104a30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2548330917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d01033c0 0x7fd0d01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 -- 172.21.15.72:0/2548330917 >> 172.21.15.72:0/2548330917 conn(0x7fd0d00fe970 msgr2=0x7fd0d0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 -- 172.21.15.72:0/2548330917 shutdown_connections 2024-03-11T19:22:07.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.553+0000 7fd0d8400640 1 -- 172.21.15.72:0/2548330917 wait complete. 2024-03-11T19:22:07.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.554+0000 7fd0d8400640 1 Processor -- start 2024-03-11T19:22:07.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.554+0000 7fd0d8400640 1 -- start start 2024-03-11T19:22:07.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.554+0000 7fd0d8400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.554+0000 7fd0d8400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01045b0 0x7fd0d0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d8400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d8400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01a0f10 con 0x7fd0d0104f70 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d8400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01a1080 con 0x7fd0d01033c0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d8400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01a1240 con 0x7fd0d01045b0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d6400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01045b0 0x7fd0d0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36804/0 (socket says 172.21.15.72:36804) 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d6400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51110/0 (socket says 172.21.15.72:51110) 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 -- 172.21.15.72:0/2861593937 learned_addr learned my addr 172.21.15.72:0/2861593937 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 -- 172.21.15.72:0/2861593937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01045b0 msgr2=0x7fd0d0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01045b0 0x7fd0d0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 -- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 msgr2=0x7fd0d01a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d5a00640 1 -- 172.21.15.72:0/2861593937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009490 con 0x7fd0d01033c0 2024-03-11T19:22:07.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.555+0000 7fd0d6400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0d5a00640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400ccc0 con 0x7fd0d01033c0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d01a1520 con 0x7fd0d01033c0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0c400ce20 con 0x7fd0d01033c0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4010430 con 0x7fd0d01033c0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.556+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01a1a00 con 0x7fd0d01033c0 2024-03-11T19:22:07.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.557+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d010a760 con 0x7fd0d01033c0 2024-03-11T19:22:07.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.559+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd0c4003fb0 con 0x7fd0d01033c0 2024-03-11T19:22:07.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.560+0000 7fd0ce000640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 0x7fd0b80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:07.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.561+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fd0c4014070 con 0x7fd0d01033c0 2024-03-11T19:22:07.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.561+0000 7fd0d5000640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 0x7fd0b80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:07.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.561+0000 7fd0d5000640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 0x7fd0b80a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc02c180 tx=0x7fd0bc034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:07.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.563+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0c4096890 con 0x7fd0d01033c0 2024-03-11T19:22:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:07 smithi106 ceph-mon[35889]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:07 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3401585199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:07 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2700332034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:07 smithi072 ceph-mon[31695]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:07.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:07 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3401585199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:07 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2700332034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:07 smithi143 ceph-mon[36013]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:07.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:07 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3401585199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:07 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2700332034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:07.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.800+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0d010edd0 con 0x7fd0d01033c0 2024-03-11T19:22:07.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.802+0000 7fd0ce000640 1 -- 172.21.15.72:0/2861593937 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fd0c4099f50 con 0x7fd0d01033c0 2024-03-11T19:22:07.806 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:07.806 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:22:07.806 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 msgr2=0x7fd0b80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 0x7fd0b80a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc02c180 tx=0x7fd0bc034c00 comp rx=0 tx=0).stop 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 msgr2=0x7fd0d0199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e9f0 tx=0x7fd0c400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 shutdown_connections 2024-03-11T19:22:07.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd0b80a1a30 0x7fd0b80a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0d0104f70 0x7fd0d01a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01045b0 0x7fd0d0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 --2- 172.21.15.72:0/2861593937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0d01033c0 0x7fd0d0199420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 >> 172.21.15.72:0/2861593937 conn(0x7fd0d00fe970 msgr2=0x7fd0d0107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 shutdown_connections 2024-03-11T19:22:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:07.804+0000 7fd0d8400640 1 -- 172.21.15.72:0/2861593937 wait complete. 2024-03-11T19:22:07.826 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:08.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 -- 172.21.15.72:0/750962481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c1089b0 msgr2=0x7efc5c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 --2- 172.21.15.72:0/750962481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c1089b0 0x7efc5c108e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc48009900 tx=0x7efc4802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 -- 172.21.15.72:0/750962481 shutdown_connections 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 --2- 172.21.15.72:0/750962481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c109370 0x7efc5c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 --2- 172.21.15.72:0/750962481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c1089b0 0x7efc5c108e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 --2- 172.21.15.72:0/750962481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 -- 172.21.15.72:0/750962481 >> 172.21.15.72:0/750962481 conn(0x7efc5c076790 msgr2=0x7efc5c078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 -- 172.21.15.72:0/750962481 shutdown_connections 2024-03-11T19:22:08.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.023+0000 7efc62000640 1 -- 172.21.15.72:0/750962481 wait complete. 2024-03-11T19:22:08.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.024+0000 7efc62000640 1 Processor -- start 2024-03-11T19:22:08.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.024+0000 7efc62000640 1 -- start start 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c1089b0 0x7efc5c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 0x7efc5c1a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1a5300 con 0x7efc5c109370 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1a5470 con 0x7efc5c1077c0 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc62000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1a5630 con 0x7efc5c1089b0 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 0x7efc5c1a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36822/0 (socket says 172.21.15.72:36822) 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 -- 172.21.15.72:0/3986445413 learned_addr learned my addr 172.21.15.72:0/3986445413 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5aa00640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c1089b0 0x7efc5c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 -- 172.21.15.72:0/3986445413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c1089b0 msgr2=0x7efc5c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c1089b0 0x7efc5c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 -- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 msgr2=0x7efc5c1a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 0x7efc5c1a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5b400640 1 -- 172.21.15.72:0/3986445413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48009560 con 0x7efc5c1077c0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.025+0000 7efc5be00640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 0x7efc5c1a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc5b400640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e9c0 tx=0x7efc4c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c009700 con 0x7efc5c1077c0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc4c00cd80 con 0x7efc5c1077c0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c1a5910 con 0x7efc5c1077c0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c010430 con 0x7efc5c1077c0 2024-03-11T19:22:08.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.026+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c1a5df0 con 0x7efc5c1077c0 2024-03-11T19:22:08.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.027+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c10eae0 con 0x7efc5c1077c0 2024-03-11T19:22:08.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.029+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efc4c003680 con 0x7efc5c1077c0 2024-03-11T19:22:08.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.030+0000 7efc53e00640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 0x7efc400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.031+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7efc4c014070 con 0x7efc5c1077c0 2024-03-11T19:22:08.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.031+0000 7efc5aa00640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 0x7efc400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.031+0000 7efc5aa00640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 0x7efc400a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc48009230 tx=0x7efc48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.035+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc4c0969c0 con 0x7efc5c1077c0 2024-03-11T19:22:08.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.230+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc5c02ce70 con 0x7efc5c1077c0 2024-03-11T19:22:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.233+0000 7efc53e00640 1 -- 172.21.15.72:0/3986445413 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7efc4c09a080 con 0x7efc5c1077c0 2024-03-11T19:22:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:08.237 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 msgr2=0x7efc400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 0x7efc400a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc48009230 tx=0x7efc48005fd0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 msgr2=0x7efc5c19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e9c0 tx=0x7efc4c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 shutdown_connections 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7efc400a1a30 0x7efc400a3ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc5c109370 0x7efc5c1a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc5c1089b0 0x7efc5c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 --2- 172.21.15.72:0/3986445413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc5c1077c0 0x7efc5c19d810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 >> 172.21.15.72:0/3986445413 conn(0x7efc5c076790 msgr2=0x7efc5c10bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 shutdown_connections 2024-03-11T19:22:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.235+0000 7efc62000640 1 -- 172.21.15.72:0/3986445413 wait complete. 2024-03-11T19:22:08.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:22:08.258 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 -- 172.21.15.72:0/933688838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0109350 msgr2=0x7f3ad010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 --2- 172.21.15.72:0/933688838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0109350 0x7f3ad010fbe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009be0 tx=0x7f3ac802bef0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 -- 172.21.15.72:0/933688838 shutdown_connections 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 --2- 172.21.15.72:0/933688838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0109350 0x7f3ad010fbe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 --2- 172.21.15.72:0/933688838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0108990 0x7f3ad0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 --2- 172.21.15.72:0/933688838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 0x7f3ad0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 -- 172.21.15.72:0/933688838 >> 172.21.15.72:0/933688838 conn(0x7f3ad0076740 msgr2=0x7f3ad0078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.462+0000 7f3ad6000640 1 -- 172.21.15.72:0/933688838 shutdown_connections 2024-03-11T19:22:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.463+0000 7f3ad6000640 1 -- 172.21.15.72:0/933688838 wait complete. 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 Processor -- start 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 -- start start 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 0x7f3ad019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0108990 0x7f3ad019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01a52b0 con 0x7f3ad0109350 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01a5420 con 0x7f3ad0108990 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3ad6000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01a55e0 con 0x7f3ad01077a0 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.464+0000 7f3acfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51144/0 (socket says 172.21.15.72:51144) 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 -- 172.21.15.72:0/2003207868 learned_addr learned my addr 172.21.15.72:0/2003207868 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:08.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acea00640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0108990 0x7f3ad019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acf400640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 0x7f3ad019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 -- 172.21.15.72:0/2003207868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 msgr2=0x7f3ad019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 0x7f3ad019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 -- 172.21.15.72:0/2003207868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0108990 msgr2=0x7f3ad019dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0108990 0x7f3ad019dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 -- 172.21.15.72:0/2003207868 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0009560 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3acfe00640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004af0 tx=0x7f3ac8033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.465+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8003d80 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.466+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ac802ce40 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.466+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac8009840 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.466+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8034960 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.466+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad01a5b80 con 0x7f3ad0109350 2024-03-11T19:22:08.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.467+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad010eac0 con 0x7f3ad0109350 2024-03-11T19:22:08.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.468+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3ac8034ac0 con 0x7f3ad0109350 2024-03-11T19:22:08.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.470+0000 7f3ac7e00640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 0x7f3ab40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.470+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f3ac80efe80 con 0x7f3ad0109350 2024-03-11T19:22:08.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.470+0000 7f3acf400640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 0x7f3ab40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.471+0000 7f3acf400640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 0x7f3ab40a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009530 tx=0x7f3ac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.474+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ac80b8d60 con 0x7f3ad0109350 2024-03-11T19:22:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2957630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2861593937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3986445413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.668+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ad002ce40 con 0x7f3ad0109350 2024-03-11T19:22:08.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.669+0000 7f3ac7e00640 1 -- 172.21.15.72:0/2003207868 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ac80bc420 con 0x7f3ad0109350 2024-03-11T19:22:08.673 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:22:08.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 msgr2=0x7f3ab40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 0x7f3ab40a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009530 tx=0x7f3ac0009260 comp rx=0 tx=0).stop 2024-03-11T19:22:08.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 msgr2=0x7f3ad01a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004af0 tx=0x7f3ac8033d00 comp rx=0 tx=0).stop 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 shutdown_connections 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f3ab40a1a30 0x7f3ab40a3ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ad0109350 0x7f3ad01a4d70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0108990 0x7f3ad019dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 --2- 172.21.15.72:0/2003207868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad01077a0 0x7f3ad019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 >> 172.21.15.72:0/2003207868 conn(0x7f3ad0076740 msgr2=0x7f3ad010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 shutdown_connections 2024-03-11T19:22:08.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.672+0000 7f3ad6000640 1 -- 172.21.15.72:0/2003207868 wait complete. 2024-03-11T19:22:08.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2957630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:08.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2861593937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3986445413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2957630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:08.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2861593937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3986445413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:08.697 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:22:08.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 -- 172.21.15.72:0/3247636920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0077720 msgr2=0x7f82b0077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 --2- 172.21.15.72:0/3247636920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0077720 0x7f82b0077fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a802c100 comp rx=0 tx=0).stop 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 -- 172.21.15.72:0/3247636920 shutdown_connections 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 --2- 172.21.15.72:0/3247636920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0077720 0x7f82b0077fc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 --2- 172.21.15.72:0/3247636920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 0x7f82b00771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 --2- 172.21.15.72:0/3247636920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0075b40 0x7f82b0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 -- 172.21.15.72:0/3247636920 >> 172.21.15.72:0/3247636920 conn(0x7f82b00fe930 msgr2=0x7f82b0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 -- 172.21.15.72:0/3247636920 shutdown_connections 2024-03-11T19:22:08.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.908+0000 7f82b4c00640 1 -- 172.21.15.72:0/3247636920 wait complete. 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.909+0000 7f82b4c00640 1 Processor -- start 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.909+0000 7f82b4c00640 1 -- start start 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.909+0000 7f82b4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82b4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 0x7f82b019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82b4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0077720 0x7f82b01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82b4c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a52a0 con 0x7f82b0075b40 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82b4c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a5410 con 0x7f82b0077720 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82b4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a55d0 con 0x7f82b0076d80 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51164/0 (socket says 172.21.15.72:51164) 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 -- 172.21.15.72:0/832932269 learned_addr learned my addr 172.21.15.72:0/832932269 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82a5400640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 0x7f82b019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ae800640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0077720 0x7f82b01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 -- 172.21.15.72:0/832932269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 msgr2=0x7f82b019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 0x7f82b019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 -- 172.21.15.72:0/832932269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0077720 msgr2=0x7f82b01a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0077720 0x7f82b01a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 -- 172.21.15.72:0/832932269 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0009490 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.910+0000 7f82ade00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82a000ea60 tx=0x7f82a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.911+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000cde0 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.911+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a00035f0 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.911+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a8009900 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.911+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0010690 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.911+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01a5c30 con 0x7f82b0075b40 2024-03-11T19:22:08.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.912+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b010fd30 con 0x7f82b0075b40 2024-03-11T19:22:08.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.914+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f82a0003af0 con 0x7f82b0075b40 2024-03-11T19:22:08.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.915+0000 7f82a7400640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 0x7f82940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:08.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.915+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f82a0014070 con 0x7f82b0075b40 2024-03-11T19:22:08.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.915+0000 7f82a5400640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 0x7f82940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:08.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.916+0000 7f82a5400640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 0x7f82940a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82b019ecc0 tx=0x7f8298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:08.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:08.919+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82a0096a80 con 0x7f82b0075b40 2024-03-11T19:22:09.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.099+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82b002ceb0 con 0x7f82b0075b40 2024-03-11T19:22:09.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.100+0000 7f82a7400640 1 -- 172.21.15.72:0/832932269 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82a009a140 con 0x7f82b0075b40 2024-03-11T19:22:09.104 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:09.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 msgr2=0x7f82940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 0x7f82940a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82b019ecc0 tx=0x7f8298005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:09.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 msgr2=0x7f82b019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82a000ea60 tx=0x7f82a000ef20 comp rx=0 tx=0).stop 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 shutdown_connections 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f82940a1a30 0x7f82940a3ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0077720 0x7f82b01a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b0076d80 0x7f82b019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 --2- 172.21.15.72:0/832932269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82b0075b40 0x7f82b019d7b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 >> 172.21.15.72:0/832932269 conn(0x7f82b00fe930 msgr2=0x7f82b0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 shutdown_connections 2024-03-11T19:22:09.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.103+0000 7f82b4c00640 1 -- 172.21.15.72:0/832932269 wait complete. 2024-03-11T19:22:09.124 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:22:09.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.333+0000 7f6a83e00640 1 -- 172.21.15.72:0/2771681899 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c032ab0 con 0x7f6a84077720 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 -- 172.21.15.72:0/2771681899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 msgr2=0x7f6a84077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 --2- 172.21.15.72:0/2771681899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a84077fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009ca0 tx=0x7f6a7c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 -- 172.21.15.72:0/2771681899 shutdown_connections 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 --2- 172.21.15.72:0/2771681899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a84077fc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 --2- 172.21.15.72:0/2771681899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84076d80 0x7f6a840771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 --2- 172.21.15.72:0/2771681899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84075b40 0x7f6a84075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 -- 172.21.15.72:0/2771681899 >> 172.21.15.72:0/2771681899 conn(0x7f6a840fe930 msgr2=0x7f6a84100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 -- 172.21.15.72:0/2771681899 shutdown_connections 2024-03-11T19:22:09.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.334+0000 7f6a8c200640 1 -- 172.21.15.72:0/2771681899 wait complete. 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 Processor -- start 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 -- start start 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84076d80 0x7f6a8419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a841a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a841a5310 con 0x7f6a84077720 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a841a5480 con 0x7f6a84076d80 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.335+0000 7f6a8c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a841a5640 con 0x7f6a84075b40 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a8a200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a841a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a88e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84076d80 0x7f6a8419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:48194/0 (socket says 172.21.15.72:48194) 2024-03-11T19:22:09.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 -- 172.21.15.72:0/3827684760 learned_addr learned my addr 172.21.15.72:0/3827684760 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 -- 172.21.15.72:0/3827684760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84076d80 msgr2=0x7f6a8419dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84076d80 0x7f6a8419dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 -- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 msgr2=0x7f6a841a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a841a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 -- 172.21.15.72:0/3827684760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009900 con 0x7f6a84075b40 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a8a200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a841a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.336+0000 7f6a89800640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400e9f0 tx=0x7f6a7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.337+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74009700 con 0x7f6a84075b40 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.337+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a841a5920 con 0x7f6a84075b40 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.337+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a7400cd80 con 0x7f6a84075b40 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.337+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74018920 con 0x7f6a84075b40 2024-03-11T19:22:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.337+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a841a5e80 con 0x7f6a84075b40 2024-03-11T19:22:09.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.338+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8410fe70 con 0x7f6a84075b40 2024-03-11T19:22:09.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.340+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6a74018a80 con 0x7f6a84075b40 2024-03-11T19:22:09.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.341+0000 7f6a82000640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 0x7f6a680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.341+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6a74014070 con 0x7f6a84075b40 2024-03-11T19:22:09.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.342+0000 7f6a88e00640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 0x7f6a680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.342+0000 7f6a88e00640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 0x7f6a680a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a8419ed30 tx=0x7f6a70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.346+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a74052d20 con 0x7f6a84075b40 2024-03-11T19:22:09.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:09 smithi072 ceph-mon[31695]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:09.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:09 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2003207868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:09 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/832932269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.543+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a8402ce40 con 0x7f6a84075b40 2024-03-11T19:22:09.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.544+0000 7f6a82000640 1 -- 172.21.15.72:0/3827684760 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a74096a30 con 0x7f6a84075b40 2024-03-11T19:22:09.548 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:22:09.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 msgr2=0x7f6a680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 0x7f6a680a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a8419ed30 tx=0x7f6a70005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 msgr2=0x7f6a8419d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400e9f0 tx=0x7f6a7400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 shutdown_connections 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6a680a1a30 0x7f6a680a3ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a84077720 0x7f6a841a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a84076d80 0x7f6a8419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 --2- 172.21.15.72:0/3827684760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a84075b40 0x7f6a8419d820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 >> 172.21.15.72:0/3827684760 conn(0x7f6a840fe930 msgr2=0x7f6a84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:09.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 shutdown_connections 2024-03-11T19:22:09.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.547+0000 7f6a8c200640 1 -- 172.21.15.72:0/3827684760 wait complete. 2024-03-11T19:22:09.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:22:09.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:09 smithi106 ceph-mon[35889]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:09.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:09 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2003207868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:09 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/832932269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:09 smithi143 ceph-mon[36013]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:09.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:09 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2003207868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:09 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/832932269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 -- 172.21.15.72:0/2669469118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c076d80 msgr2=0x7fa74c0771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 --2- 172.21.15.72:0/2669469118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c076d80 0x7fa74c0771e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa738009900 tx=0x7fa73802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 -- 172.21.15.72:0/2669469118 shutdown_connections 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 --2- 172.21.15.72:0/2669469118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c077720 0x7fa74c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 --2- 172.21.15.72:0/2669469118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c076d80 0x7fa74c0771e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 --2- 172.21.15.72:0/2669469118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 0x7fa74c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.782+0000 7fa754600640 1 -- 172.21.15.72:0/2669469118 >> 172.21.15.72:0/2669469118 conn(0x7fa74c0fe930 msgr2=0x7fa74c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.783+0000 7fa754600640 1 -- 172.21.15.72:0/2669469118 shutdown_connections 2024-03-11T19:22:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.783+0000 7fa754600640 1 -- 172.21.15.72:0/2669469118 wait complete. 2024-03-11T19:22:09.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 Processor -- start 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 -- start start 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 0x7fa74c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c076d80 0x7fa74c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c1a5320 con 0x7fa74c077720 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c1a5490 con 0x7fa74c075b40 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa754600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c1a5650 con 0x7fa74c076d80 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa752600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa752600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51188/0 (socket says 172.21.15.72:51188) 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa752600640 1 -- 172.21.15.72:0/3713568764 learned_addr learned my addr 172.21.15.72:0/3713568764 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa751c00640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 0x7fa74c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.784+0000 7fa751200640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c076d80 0x7fa74c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 -- 172.21.15.72:0/3713568764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c076d80 msgr2=0x7fa74c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c076d80 0x7fa74c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 -- 172.21.15.72:0/3713568764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 msgr2=0x7fa74c19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 0x7fa74c19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 -- 172.21.15.72:0/3713568764 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738009560 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa752600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa74400eb10 tx=0x7fa74400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744004b20 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa744004c80 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c1a5930 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.785+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744010650 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.786+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c1a5e60 con 0x7fa74c077720 2024-03-11T19:22:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.787+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c10fe90 con 0x7fa74c077720 2024-03-11T19:22:09.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.789+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa744005020 con 0x7fa74c077720 2024-03-11T19:22:09.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.790+0000 7fa74a000640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 0x7fa7340a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:09.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.790+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa744014070 con 0x7fa74c077720 2024-03-11T19:22:09.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.791+0000 7fa751c00640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 0x7fa7340a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:09.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.791+0000 7fa751c00640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 0x7fa7340a3f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa74000bbe0 tx=0x7fa74000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:09.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.794+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa744096ae0 con 0x7fa74c077720 2024-03-11T19:22:09.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.992+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa74c02cea0 con 0x7fa74c077720 2024-03-11T19:22:09.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.992+0000 7fa74a000640 1 -- 172.21.15.72:0/3713568764 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa74409a1a0 con 0x7fa74c077720 2024-03-11T19:22:09.997 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 msgr2=0x7fa7340a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 0x7fa7340a3f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa74000bbe0 tx=0x7fa74000b390 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 msgr2=0x7fa74c1a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa74400eb10 tx=0x7fa74400c2d0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 shutdown_connections 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa7340a1ab0 0x7fa7340a3f70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa74c077720 0x7fa74c1a4de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa74c076d80 0x7fa74c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 --2- 172.21.15.72:0/3713568764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c075b40 0x7fa74c19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 >> 172.21.15.72:0/3713568764 conn(0x7fa74c0fe930 msgr2=0x7fa74c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.995+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 shutdown_connections 2024-03-11T19:22:10.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:09.996+0000 7fa754600640 1 -- 172.21.15.72:0/3713568764 wait complete. 2024-03-11T19:22:10.019 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 -- 172.21.15.72:0/3618920469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181087d0 msgr2=0x7f1018108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 --2- 172.21.15.72:0/3618920469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181087d0 0x7f1018108c50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 -- 172.21.15.72:0/3618920469 shutdown_connections 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 --2- 172.21.15.72:0/3618920469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1018109190 0x7f101810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 --2- 172.21.15.72:0/3618920469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181087d0 0x7f1018108c50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 --2- 172.21.15.72:0/3618920469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10181075e0 0x7f1018107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 -- 172.21.15.72:0/3618920469 >> 172.21.15.72:0/3618920469 conn(0x7f1018076590 msgr2=0x7f10180789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.222+0000 7f1017e00640 1 -- 172.21.15.72:0/3618920469 shutdown_connections 2024-03-11T19:22:10.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.223+0000 7f1017e00640 1 -- 172.21.15.72:0/3618920469 wait complete. 2024-03-11T19:22:10.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.223+0000 7f1017e00640 1 Processor -- start 2024-03-11T19:22:10.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 -- start start 2024-03-11T19:22:10.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10181087d0 0x7f101819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1018109190 0x7f10181a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a5120 con 0x7f10181075e0 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a5290 con 0x7f10181087d0 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a5450 con 0x7f1018109190 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1016a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1017400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1018109190 0x7f10181a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1016a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51202/0 (socket says 172.21.15.72:51202) 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f1016a00640 1 -- 172.21.15.72:0/3051343497 learned_addr learned my addr 172.21.15.72:0/3051343497 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.224+0000 7f100fe00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10181087d0 0x7f101819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 -- 172.21.15.72:0/3051343497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1018109190 msgr2=0x7f10181a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1018109190 0x7f10181a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 -- 172.21.15.72:0/3051343497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10181087d0 msgr2=0x7f101819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10181087d0 0x7f101819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 -- 172.21.15.72:0/3051343497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000009560 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1016a00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f100800e9c0 tx=0x7f100800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080096e0 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f100800cd80 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008010430 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10181a5730 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.225+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10181a5c10 con 0x7f10181075e0 2024-03-11T19:22:10.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.226+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101810e8a0 con 0x7f10181075e0 2024-03-11T19:22:10.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.228+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1008003680 con 0x7f10181075e0 2024-03-11T19:22:10.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.230+0000 7f100f400640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 0x7f0ffc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.230+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f1008014070 con 0x7f10181075e0 2024-03-11T19:22:10.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.231+0000 7f100fe00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 0x7f0ffc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.231+0000 7f100fe00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 0x7f0ffc0a3f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f100002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:10.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.235+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1008096940 con 0x7f10181075e0 2024-03-11T19:22:10.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.471+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1018113040 con 0x7f10181075e0 2024-03-11T19:22:10.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.473+0000 7f100f400640 1 -- 172.21.15.72:0/3051343497 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f100809a000 con 0x7f10181075e0 2024-03-11T19:22:10.477 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:10.477 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:22:10.478 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:10.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 msgr2=0x7f0ffc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 0x7f0ffc0a3f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f100002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 msgr2=0x7f101819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f100800e9c0 tx=0x7f100800ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 shutdown_connections 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0ffc0a1ab0 0x7f0ffc0a3f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1018109190 0x7f10181a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10181087d0 0x7f101819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.475+0000 7f1017e00640 1 --2- 172.21.15.72:0/3051343497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10181075e0 0x7f101819d630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.476+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 >> 172.21.15.72:0/3051343497 conn(0x7f1018076590 msgr2=0x7f101810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.476+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 shutdown_connections 2024-03-11T19:22:10.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.476+0000 7f1017e00640 1 -- 172.21.15.72:0/3051343497 wait complete. 2024-03-11T19:22:10.499 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:10.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:10 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3827684760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:10 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3713568764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:10 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3827684760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:10 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3713568764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:10 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3827684760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:10 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3713568764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:10.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 -- 172.21.15.72:0/1756183341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 msgr2=0x7f00a81037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 --2- 172.21.15.72:0/1756183341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a81037e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0098009860 tx=0x7f009802bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:10.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 -- 172.21.15.72:0/1756183341 shutdown_connections 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 --2- 172.21.15.72:0/1756183341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 --2- 172.21.15.72:0/1756183341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 0x7f00a8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 --2- 172.21.15.72:0/1756183341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a81037e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 -- 172.21.15.72:0/1756183341 >> 172.21.15.72:0/1756183341 conn(0x7f00a80fe970 msgr2=0x7f00a8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 -- 172.21.15.72:0/1756183341 shutdown_connections 2024-03-11T19:22:10.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.707+0000 7f00ada00640 1 -- 172.21.15.72:0/1756183341 wait complete. 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.708+0000 7f00ada00640 1 Processor -- start 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.708+0000 7f00ada00640 1 -- start start 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a8199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 0x7f00a8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a81a0fb0 con 0x7f00a8104f70 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a81a1120 con 0x7f00a81033c0 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00ada00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a81a12e0 con 0x7f00a81045b0 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51216/0 (socket says 172.21.15.72:51216) 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 -- 172.21.15.72:0/3647810778 learned_addr learned my addr 172.21.15.72:0/3647810778 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a6a00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a8199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a6000640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 0x7f00a8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 -- 172.21.15.72:0/3647810778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 msgr2=0x7f00a8199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 0x7f00a8199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 -- 172.21.15.72:0/3647810778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 msgr2=0x7f00a8199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a8199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.709+0000 7f00a7400640 1 -- 172.21.15.72:0/3647810778 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0009560 con 0x7f00a8104f70 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f00a7400640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e940 tx=0x7f00a000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000cd80 con 0x7f00a8104f70 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00a00035f0 con 0x7f00a8104f70 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0010690 con 0x7f00a8104f70 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0098009490 con 0x7f00a8104f70 2024-03-11T19:22:10.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.710+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a81a18f0 con 0x7f00a8104f70 2024-03-11T19:22:10.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.711+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a810a700 con 0x7f00a8104f70 2024-03-11T19:22:10.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.713+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f00a0003af0 con 0x7f00a8104f70 2024-03-11T19:22:10.718 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.715+0000 7f009f400640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 0x7f008c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:10.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.715+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f00a0014070 con 0x7f00a8104f70 2024-03-11T19:22:10.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.715+0000 7f00a6a00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 0x7f008c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:10.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.716+0000 7f00a6a00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 0x7f008c0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0098034b10 tx=0x7f0098034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:10.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.717+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00a0096bd0 con 0x7f00a8104f70 2024-03-11T19:22:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.919+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00a802ce40 con 0x7f00a8104f70 2024-03-11T19:22:10.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.922+0000 7f009f400640 1 -- 172.21.15.72:0/3647810778 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7f00a009a290 con 0x7f00a8104f70 2024-03-11T19:22:10.927 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:10.927 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 msgr2=0x7f008c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 0x7f008c0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0098034b10 tx=0x7f0098034be0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 msgr2=0x7f00a81a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e940 tx=0x7f00a000ee00 comp rx=0 tx=0).stop 2024-03-11T19:22:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 shutdown_connections 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f008c0a1a30 0x7f008c0a3ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00a8104f70 0x7f00a81a0a70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00a81045b0 0x7f00a8199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 --2- 172.21.15.72:0/3647810778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00a81033c0 0x7f00a8199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 >> 172.21.15.72:0/3647810778 conn(0x7f00a80fe970 msgr2=0x7f00a81078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 shutdown_connections 2024-03-11T19:22:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:10.925+0000 7f00ada00640 1 -- 172.21.15.72:0/3647810778 wait complete. 2024-03-11T19:22:10.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:22:11.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.156+0000 7f84a5a00640 1 -- 172.21.15.72:0/1795528279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01077c0 msgr2=0x7f84a0107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.156+0000 7f84a5a00640 1 --2- 172.21.15.72:0/1795528279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01077c0 0x7f84a0107be0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8490009ca0 tx=0x7f849002c100 comp rx=0 tx=0).stop 2024-03-11T19:22:11.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 -- 172.21.15.72:0/1795528279 shutdown_connections 2024-03-11T19:22:11.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 --2- 172.21.15.72:0/1795528279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a0109370 0x7f84a010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 --2- 172.21.15.72:0/1795528279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a01089b0 0x7f84a0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 --2- 172.21.15.72:0/1795528279 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01077c0 0x7f84a0107be0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 -- 172.21.15.72:0/1795528279 >> 172.21.15.72:0/1795528279 conn(0x7f84a0076790 msgr2=0x7f84a0078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 -- 172.21.15.72:0/1795528279 shutdown_connections 2024-03-11T19:22:11.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.157+0000 7f84a5a00640 1 -- 172.21.15.72:0/1795528279 wait complete. 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 Processor -- start 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 -- start start 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a01077c0 0x7f84a019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 0x7f84a01a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01a5330 con 0x7f84a01089b0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01a54a0 con 0x7f84a0109370 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.158+0000 7f84a5a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01a5660 con 0x7f84a01077c0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51238/0 (socket says 172.21.15.72:51238) 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 -- 172.21.15.72:0/543051088 learned_addr learned my addr 172.21.15.72:0/543051088 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849ea00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a01077c0 0x7f84a019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849f400640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 0x7f84a01a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 -- 172.21.15.72:0/543051088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a01077c0 msgr2=0x7f84a019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a01077c0 0x7f84a019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 -- 172.21.15.72:0/543051088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 msgr2=0x7f84a01a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 0x7f84a01a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849e000640 1 -- 172.21.15.72:0/543051088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490009900 con 0x7f84a01089b0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.159+0000 7f849f400640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 0x7f84a01a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f849e000640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f848c00e9f0 tx=0x7f848c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c009700 con 0x7f84a01089b0 2024-03-11T19:22:11.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f848c00cd80 con 0x7f84a01089b0 2024-03-11T19:22:11.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a01a5940 con 0x7f84a01089b0 2024-03-11T19:22:11.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c018920 con 0x7f84a01089b0 2024-03-11T19:22:11.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.160+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a01a5ea0 con 0x7f84a01089b0 2024-03-11T19:22:11.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.161+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a010eae0 con 0x7f84a01089b0 2024-03-11T19:22:11.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.164+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f848c018a80 con 0x7f84a01089b0 2024-03-11T19:22:11.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.165+0000 7f8497400640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 0x7f84840a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.166+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f848c014070 con 0x7f84a01089b0 2024-03-11T19:22:11.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.166+0000 7f849ea00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 0x7f84840a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.167+0000 7f849ea00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 0x7f84840a3f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f849002c710 tx=0x7f84900326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.167+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f848c096e10 con 0x7f84a01089b0 2024-03-11T19:22:11.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.395+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84a01131c0 con 0x7f84a01089b0 2024-03-11T19:22:11.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.397+0000 7f8497400640 1 -- 172.21.15.72:0/543051088 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f848c026020 con 0x7f84a01089b0 2024-03-11T19:22:11.403 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:11.403 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:22:11.403 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 msgr2=0x7f84840a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 0x7f84840a3f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f849002c710 tx=0x7f84900326d0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 msgr2=0x7f84a019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f848c00e9f0 tx=0x7f848c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 shutdown_connections 2024-03-11T19:22:11.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f84840a1ab0 0x7f84840a3f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84a0109370 0x7f84a01a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a01089b0 0x7f84a019dd80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 --2- 172.21.15.72:0/543051088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a01077c0 0x7f84a019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 >> 172.21.15.72:0/543051088 conn(0x7f84a0076790 msgr2=0x7f84a010bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 shutdown_connections 2024-03-11T19:22:11.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.400+0000 7f84a5a00640 1 -- 172.21.15.72:0/543051088 wait complete. 2024-03-11T19:22:11.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:19:41.837094+0000', 'modified': '2024-03-11T19:20:15.233431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24542, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/2743792791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2743792791}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2743792791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/902566146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 902566146}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 902566146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/2982797341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2982797341}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2982797341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.143:6839/3455090526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 3455090526}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 3455090526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.72:6839/3478523060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 3478523060}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 3478523060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/806155003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 806155003}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 806155003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:22:11.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:22:11.423 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 -- 172.21.15.72:0/1119696140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77001077a0 msgr2=0x7f7700107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 --2- 172.21.15.72:0/1119696140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77001077a0 0x7f7700107bc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009900 tx=0x7f76f402bc40 comp rx=0 tx=0).stop 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 -- 172.21.15.72:0/1119696140 shutdown_connections 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 --2- 172.21.15.72:0/1119696140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7700109350 0x7f770010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 --2- 172.21.15.72:0/1119696140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 0x7f7700108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 --2- 172.21.15.72:0/1119696140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77001077a0 0x7f7700107bc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 -- 172.21.15.72:0/1119696140 >> 172.21.15.72:0/1119696140 conn(0x7f7700076740 msgr2=0x7f7700078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 -- 172.21.15.72:0/1119696140 shutdown_connections 2024-03-11T19:22:11.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.626+0000 7f7707a00640 1 -- 172.21.15.72:0/1119696140 wait complete. 2024-03-11T19:22:11.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.627+0000 7f7707a00640 1 Processor -- start 2024-03-11T19:22:11.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.627+0000 7f7707a00640 1 -- start start 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 0x7f770019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 0x7f770019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a5340 con 0x7f7700109350 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a54b0 con 0x7f77001077a0 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7707a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001a5670 con 0x7f7700108990 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 0x7f770019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51254/0 (socket says 172.21.15.72:51254) 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705a00640 1 -- 172.21.15.72:0/3918700537 learned_addr learned my addr 172.21.15.72:0/3918700537 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f76ffe00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 0x7f770019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 -- 172.21.15.72:0/3918700537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 msgr2=0x7f770019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 0x7f770019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 -- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 msgr2=0x7f77001a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705000640 1 -- 172.21.15.72:0/3918700537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4009560 con 0x7f77001077a0 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.628+0000 7f7705a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f7705000640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 0x7f770019d850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036040 tx=0x7f76f4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f76fd600640 1 -- 172.21.15.72:0/3918700537 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40043d0 con 0x7f77001077a0 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f76fd600640 1 -- 172.21.15.72:0/3918700537 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f40052d0 con 0x7f77001077a0 2024-03-11T19:22:11.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001a58f0 con 0x7f77001077a0 2024-03-11T19:22:11.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f76fd600640 1 -- 172.21.15.72:0/3918700537 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4003cd0 con 0x7f77001077a0 2024-03-11T19:22:11.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.629+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a5dd0 con 0x7f77001077a0 2024-03-11T19:22:11.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 Processor -- start 2024-03-11T19:22:11.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 -- start start 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e99959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f76fd600640 1 -- 172.21.15.72:0/3918700537 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f76f4005440 con 0x7f77001077a0 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e999aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e9904b30 con 0x55b0e999b0e0 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e9902060 con 0x55b0e9995f70 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.633+0000 7f770a7a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e98fd840 con 0x55b0e9995580 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51266/0 (socket says 172.21.15.72:51266) 2024-03-11T19:22:11.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f7705000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e999aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 -- 172.21.15.72:0/2874768211 learned_addr learned my addr 172.21.15.72:0/2874768211 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f7705a00640 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e99959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 -- 172.21.15.72:0/2874768211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 msgr2=0x55b0e99959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e99959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 -- 172.21.15.72:0/2874768211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 msgr2=0x55b0e999aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e999aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 -- 172.21.15.72:0/2874768211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0e98c54c0 con 0x55b0e999b0e0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.634+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009ca0 tx=0x7f76ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f966b050a3ec3c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f76f2000640 1 -- 172.21.15.72:0/2874768211 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec019630 con 0x55b0e999b0e0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f76f2000640 1 -- 172.21.15.72:0/2874768211 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ec009e50 con 0x55b0e999b0e0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f76f2000640 1 -- 172.21.15.72:0/2874768211 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00ea80 con 0x55b0e999b0e0 2024-03-11T19:22:11.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f76fd600640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 0x7f76e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f76fd600640 1 -- 172.21.15.72:0/3918700537 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f76f40efe80 con 0x7f77001077a0 2024-03-11T19:22:11.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f770a7a0740 1 -- 172.21.15.72:0/2874768211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 msgr2=0x55b0e999d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f7705a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 0x7f76e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.635+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009ca0 tx=0x7f76ec011810 comp rx=0 tx=0).stop 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 -- 172.21.15.72:0/2874768211 shutdown_connections 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e999b0e0 0x55b0e999d570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e999aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2874768211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e99959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 -- 172.21.15.72:0/2874768211 >> 172.21.15.72:0/2874768211 conn(0x55b0e9990af0 msgr2=0x55b0e9990ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 -- 172.21.15.72:0/2874768211 shutdown_connections 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 -- 172.21.15.72:0/2874768211 wait complete. 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f7705a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 0x7f76e80a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55b0e99967d0 tx=0x7f76f8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 -- start start 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.636+0000 7f770a7a0740 1 Processor -- start 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e98a2ea0 con 0x55b0e9995f70 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e98a2b40 con 0x55b0e999b0e0 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e9645f20 con 0x55b0e9995580 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f770a7a0740 10 client.0 _ll_register_callbacks cb 0x55b0e98b6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f7705000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f7705a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f7705a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36948/0 (socket says 172.21.15.72:36948) 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f7705a00640 1 -- 172.21.15.72:0/2237416608 learned_addr learned my addr 172.21.15.72:0/2237416608 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f7705000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:48282/0 (socket says 172.21.15.72:48282) 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.637+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51282/0 (socket says 172.21.15.72:51282) 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 -- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 msgr2=0x55b0e999f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 -- 172.21.15.72:0/2237416608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 msgr2=0x55b0e999f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 -- 172.21.15.72:0/2237416608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec0098d0 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705000640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f7705a00640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f76f800fbb0 tx=0x7f76f8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4780f46fa85d06ac server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f76e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0e9921c40 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.638+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f76f8011c10 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f8011dd0 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 10 client.25142 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0e98fbec0 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b0e9921c40 con 0x55b0e999b0e0 2024-03-11T19:22:11.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76f801f6c0 con 0x55b0e999b0e0 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 10 client.25142 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 10 client.25142 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 20 client.25142 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e99aba50 0x55b0e99aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 4 client.25142 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.639+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 --> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0e95fd6c0 con 0x55b0e99aba50 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.640+0000 7f7705000640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e99aba50 0x55b0e99aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.640+0000 7f7705000640 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e99aba50 0x55b0e99aac70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b579ce79c9e1eb4 server_cookie=df8ea1c61c06ccaf in_seq=0 out_seq=0 2024-03-11T19:22:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.640+0000 7f76e6a00640 10 client.25142 ms_handle_connect on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:11.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:11 smithi106 ceph-mon[35889]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:11.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:11 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3051343497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:11 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3647810778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.641+0000 7f76e6a00640 1 -- 172.21.15.72:0/2237416608 <== mds.2 v2:172.21.15.143:6838/3455090526 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b0e95fd6c0 con 0x55b0e99aba50 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.641+0000 7f76e6a00640 10 client.25142 handle_command_reply: tid=0 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.641+0000 7f770a7a0740 1 client.25142 shutdown 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 msgr2=0x55b0e99a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f76f800fbb0 tx=0x7f76f8004ab0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 shutdown_connections 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e99aba50 0x55b0e99aac70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e99a6930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995f70 0x55b0e999f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2237416608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995580 0x55b0e999f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f76e6a00640 0 client.25142 ms_handle_reset on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:11.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 >> 172.21.15.72:0/2237416608 conn(0x55b0e9990af0 msgr2=0x55b0e998fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 shutdown_connections 2024-03-11T19:22:11.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 1 -- 172.21.15.72:0/2237416608 wait complete. 2024-03-11T19:22:11.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.642+0000 7f770a7a0740 20 client.25142 trim_cache size 0 max 16384 2024-03-11T19:22:11.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 Processor -- start 2024-03-11T19:22:11.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 -- start start 2024-03-11T19:22:11.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e99a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e95fd6c0 con 0x55b0e9995580 2024-03-11T19:22:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e9921c40 con 0x55b0e9995f70 2024-03-11T19:22:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.659+0000 7f770a7a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e99a7420 con 0x55b0e999b0e0 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:48288/0 (socket says 172.21.15.72:48288) 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 -- 172.21.15.72:0/3405833139 learned_addr learned my addr 172.21.15.72:0/3405833139 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705000640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f76ffe00640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e99a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 -- 172.21.15.72:0/3405833139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 msgr2=0x55b0e99a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e99a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 -- 172.21.15.72:0/3405833139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 msgr2=0x55b0e99b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 -- 172.21.15.72:0/3405833139 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0e98988b0 con 0x55b0e999b0e0 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705000640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f76ffe00640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e99a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.660+0000 7f7705a00640 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a010 tx=0x7f76f800f7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6138a96499111085 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f76f1600640 1 -- 172.21.15.72:0/3405833139 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8011c10 con 0x55b0e999b0e0 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f76f1600640 1 -- 172.21.15.72:0/3405833139 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f801f900 con 0x55b0e999b0e0 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f76f1600640 1 -- 172.21.15.72:0/3405833139 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8037a80 con 0x55b0e999b0e0 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 -- 172.21.15.72:0/3405833139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 msgr2=0x55b0e999f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a010 tx=0x7f76f800f7e0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 -- 172.21.15.72:0/3405833139 shutdown_connections 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e999b0e0 0x55b0e999f3b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e9995f70 0x55b0e99a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 --2- 172.21.15.72:0/3405833139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 -- 172.21.15.72:0/3405833139 >> 172.21.15.72:0/3405833139 conn(0x55b0e9990af0 msgr2=0x55b0e999e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 -- 172.21.15.72:0/3405833139 shutdown_connections 2024-03-11T19:22:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.661+0000 7f770a7a0740 1 -- 172.21.15.72:0/3405833139 wait complete. 2024-03-11T19:22:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 -- start start 2024-03-11T19:22:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 Processor -- start 2024-03-11T19:22:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995f70 0x55b0e99a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e9a01650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e99a6ae0 con 0x55b0e9995580 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e99a6c50 con 0x55b0e999b0e0 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0e9a01b90 con 0x55b0e9995f70 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.662+0000 7f770a7a0740 10 client.0 _ll_register_callbacks cb 0x7f76f8012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51300/0 (socket says 172.21.15.72:51300) 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 -- 172.21.15.72:0/2297378917 learned_addr learned my addr 172.21.15.72:0/2297378917 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995f70 0x55b0e99a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705a00640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e9a01650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 -- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995f70 msgr2=0x55b0e99a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995f70 0x55b0e99a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 -- 172.21.15.72:0/2297378917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 msgr2=0x55b0e9a01650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e9a01650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 -- 172.21.15.72:0/2297378917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0e98fbec0 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f7705000640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76f4000c00 tx=0x7f76f40b8dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fbeb0eb81bae20 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f76e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40bc3d0 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f76f40035f0 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.663+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0e99ac420 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f4003e30 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 10 client.15048 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b0e98be510 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76f409f020 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f76f403c070 con 0x55b0e9995580 2024-03-11T19:22:11.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 10 client.15048 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 20 client.15048 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e9a06950 0x55b0e9a02e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 4 client.15048 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 --> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f76ec009e50 con 0x55b0e9a06950 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.664+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e9a06950 0x55b0e9a02e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f76ffe00640 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e9a06950 0x55b0e9a02e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98ddd6a9b4fb737d server_cookie=b60024a1aecc44b5 in_seq=0 out_seq=0 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f76e6a00640 10 client.15048 ms_handle_connect on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f76e6a00640 1 -- 172.21.15.72:0/2297378917 <== mds.2 v2:172.21.15.143:6838/3455090526 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f76ec009e50 con 0x55b0e9a06950 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f76e6a00640 10 client.15048 handle_command_reply: tid=0 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f770a7a0740 1 client.15048 shutdown 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 msgr2=0x55b0e99a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.665+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76f4000c00 tx=0x7f76f40b8dd0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 shutdown_connections 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55b0e9a06950 0x55b0e9a02e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b0e999b0e0 0x55b0e9a01650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0e9995f70 0x55b0e99a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f76e6a00640 0 client.15048 ms_handle_reset on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 --2- 172.21.15.72:0/2297378917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b0e9995580 0x55b0e99a3620 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 >> 172.21.15.72:0/2297378917 conn(0x55b0e9990af0 msgr2=0x55b0e99b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 shutdown_connections 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 1 -- 172.21.15.72:0/2297378917 wait complete. 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f770a7a0740 20 client.15048 trim_cache size 0 max 16384 2024-03-11T19:22:11.675 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24532, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 117.440549691, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 166.84464857200001 2024-03-11T19:22:11.676 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:11.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 msgr2=0x7f76e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 0x7f76e80a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55b0e99967d0 tx=0x7f76f8009500 comp rx=0 tx=0).stop 2024-03-11T19:22:11.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 msgr2=0x7f770019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.666+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 0x7f770019d850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036040 tx=0x7f76f4005e80 comp rx=0 tx=0).stop 2024-03-11T19:22:11.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 shutdown_connections 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76e80a1a30 0x7f76e80a3ef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7700109350 0x7f77001a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7700108990 0x7f770019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 --2- 172.21.15.72:0/3918700537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77001077a0 0x7f770019d850 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 >> 172.21.15.72:0/3918700537 conn(0x7f7700076740 msgr2=0x7f770010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 shutdown_connections 2024-03-11T19:22:11.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.667+0000 7f7707a00640 1 -- 172.21.15.72:0/3918700537 wait complete. 2024-03-11T19:22:11.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:11 smithi072 ceph-mon[31695]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:11.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:11 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3051343497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:11 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3647810778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.688 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-11T19:22:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:11 smithi143 ceph-mon[36013]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:11 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3051343497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:11 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3647810778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 -- 172.21.15.72:0/1186256482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb4104f70 msgr2=0x7fbbb410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 --2- 172.21.15.72:0/1186256482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb4104f70 0x7fbbb410b800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02c100 comp rx=0 tx=0).stop 2024-03-11T19:22:11.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 -- 172.21.15.72:0/1186256482 shutdown_connections 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 --2- 172.21.15.72:0/1186256482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb4104f70 0x7fbbb410b800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 --2- 172.21.15.72:0/1186256482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb41045b0 0x7fbbb4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 --2- 172.21.15.72:0/1186256482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 0x7fbbb41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 -- 172.21.15.72:0/1186256482 >> 172.21.15.72:0/1186256482 conn(0x7fbbb40fe930 msgr2=0x7fbbb4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 -- 172.21.15.72:0/1186256482 shutdown_connections 2024-03-11T19:22:11.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.891+0000 7fbbba600640 1 -- 172.21.15.72:0/1186256482 wait complete. 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 Processor -- start 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 -- start start 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 0x7fbbb41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb4104f70 0x7fbbb41a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a0ed0 con 0x7fbbb41045b0 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a1040 con 0x7fbbb4104f70 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.892+0000 7fbbba600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a1200 con 0x7fbbb41033c0 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43148/0 (socket says 172.21.15.72:43148) 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 -- 172.21.15.72:0/525384907 learned_addr learned my addr 172.21.15.72:0/525384907 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb4104f70 0x7fbbb41a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbb3400640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 0x7fbbb41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 -- 172.21.15.72:0/525384907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 msgr2=0x7fbbb41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 0x7fbbb41993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 -- 172.21.15.72:0/525384907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb4104f70 msgr2=0x7fbbb41a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb4104f70 0x7fbbb41a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 -- 172.21.15.72:0/525384907 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c009490 con 0x7fbbb41045b0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.893+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00ea60 tx=0x7fbb9c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.894+0000 7fbbb0c00640 1 -- 172.21.15.72:0/525384907 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00cde0 con 0x7fbbb41045b0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.894+0000 7fbbb0c00640 1 -- 172.21.15.72:0/525384907 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c0035f0 con 0x7fbbb41045b0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.894+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac009900 con 0x7fbbb41045b0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.894+0000 7fbbb0c00640 1 -- 172.21.15.72:0/525384907 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c010690 con 0x7fbbb41045b0 2024-03-11T19:22:11.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.894+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41a1860 con 0x7fbbb41045b0 2024-03-11T19:22:11.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 Processor -- start 2024-03-11T19:22:11.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbb0c00640 1 -- 172.21.15.72:0/525384907 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbb9c003af0 con 0x7fbbb41045b0 2024-03-11T19:22:11.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 -- start start 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641d580 0x559d7641d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d764230e0 0x559d76425570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7638cb30 con 0x559d764230e0 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7638a060 con 0x559d7641d580 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbbd395740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d76385840 con 0x559d7641df70 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbaaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d764230e0 0x559d76425570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36206/0 (socket says 172.21.15.72:36206) 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbb3400640 1 -- 172.21.15.72:0/3055995571 learned_addr learned my addr 172.21.15.72:0/3055995571 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.897+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641d580 0x559d7641d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 -- 172.21.15.72:0/3055995571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641d580 msgr2=0x559d7641d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641d580 0x559d7641d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 -- 172.21.15.72:0/3055995571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d764230e0 msgr2=0x559d76425570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d764230e0 0x559d76425570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 -- 172.21.15.72:0/3055995571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d7634d4c0 con 0x559d7641df70 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb3400640 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ca0 tx=0x7fbba4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a4ee317592f131 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbbb0c00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 0x7fbb980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbba8c00640 1 -- 172.21.15.72:0/3055995571 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4004220 con 0x559d7641df70 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.898+0000 7fbba8c00640 1 -- 172.21.15.72:0/3055995571 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba4004380 con 0x559d7641df70 2024-03-11T19:22:11.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbba8c00640 1 -- 172.21.15.72:0/3055995571 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4019830 con 0x559d7641df70 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbb0c00640 1 -- 172.21.15.72:0/525384907 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fbb9c014070 con 0x7fbbb41045b0 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 0x7fbb980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 -- 172.21.15.72:0/3055995571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 msgr2=0x559d76422a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ca0 tx=0x7fbba4011810 comp rx=0 tx=0).stop 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 0x7fbb980a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02c640 tx=0x7fbbac0326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 -- 172.21.15.72:0/3055995571 shutdown_connections 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d764230e0 0x559d76425570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641df70 0x559d76422a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 --2- 172.21.15.72:0/3055995571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641d580 0x559d7641d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.899+0000 7fbbbd395740 1 -- 172.21.15.72:0/3055995571 >> 172.21.15.72:0/3055995571 conn(0x559d76418af0 msgr2=0x559d76418ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.900+0000 7fbbbd395740 1 -- 172.21.15.72:0/3055995571 shutdown_connections 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.900+0000 7fbbbd395740 1 -- 172.21.15.72:0/3055995571 wait complete. 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.900+0000 7fbbbd395740 1 -- start start 2024-03-11T19:22:11.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.900+0000 7fbbbd395740 1 Processor -- start 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 0x559d764273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76427920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d764230e0 0x559d7642e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7632aea0 con 0x559d7641d580 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7632ab40 con 0x559d7641df70 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d760cdf20 con 0x559d764230e0 2024-03-11T19:22:11.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbbd395740 10 client.0 _ll_register_callbacks cb 0x559d7633e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 0x559d764273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d764230e0 0x559d7642e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 0x559d764273e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43158/0 (socket says 172.21.15.72:43158) 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbb3400640 1 -- 172.21.15.72:0/2066661794 learned_addr learned my addr 172.21.15.72:0/2066661794 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76427920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 -- 172.21.15.72:0/2066661794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d764230e0 msgr2=0x559d7642e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d764230e0 0x559d7642e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 -- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 msgr2=0x559d764273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 0x559d764273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.901+0000 7fbbaaa00640 1 -- 172.21.15.72:0/2066661794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba40098d0 con 0x559d7641df70 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76427920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cb3e0 tx=0x7fbb9c096440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d032e0733793204a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbba1600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c09a160 con 0x559d7641df70 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fbb9c09af00 con 0x559d7641df70 2024-03-11T19:22:11.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d763a9c40 con 0x559d7641df70 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb9c09a800 con 0x559d7641df70 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbbbd395740 10 client.25160 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d76383ec0 con 0x559d7641df70 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.902+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d763a9c40 con 0x559d7641df70 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbb9c0d5070 con 0x559d7641df70 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 10 client.25160 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 10 client.25160 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 20 client.25160 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d76433930 0x559d76437db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 4 client.25160 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 --> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d760856c0 con 0x559d76433930 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.903+0000 7fbbb3400640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d76433930 0x559d76437db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.904+0000 7fbbb3400640 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d76433930 0x559d76437db0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53362acc45906091 server_cookie=75e96bf807badee1 in_seq=0 out_seq=0 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.904+0000 7fbba1600640 10 client.25160 ms_handle_connect on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:11.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.904+0000 7fbba1600640 1 -- 172.21.15.72:0/2066661794 <== mds.1 v2:172.21.15.72:6838/3478523060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d760856c0 con 0x559d76433930 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.904+0000 7fbba1600640 10 client.25160 handle_command_reply: tid=0 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.904+0000 7fbbbd395740 1 client.25160 shutdown 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 msgr2=0x559d76427920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76427920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cb3e0 tx=0x7fbb9c096440 comp rx=0 tx=0).stop 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 shutdown_connections 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d76433930 0x559d76437db0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d764230e0 0x559d7642e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76427920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbba1600640 0 client.25160 ms_handle_reset on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 --2- 172.21.15.72:0/2066661794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d7641d580 0x559d764273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 >> 172.21.15.72:0/2066661794 conn(0x559d76418af0 msgr2=0x559d76417e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 shutdown_connections 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 1 -- 172.21.15.72:0/2066661794 wait complete. 2024-03-11T19:22:11.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.905+0000 7fbbbd395740 20 client.25160 trim_cache size 0 max 16384 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 Processor -- start 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 -- start start 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76417580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7642f220 con 0x559d76418af0 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d76419330 con 0x559d7641df70 2024-03-11T19:22:11.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbbd395740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d76425700 con 0x559d7641d580 2024-03-11T19:22:11.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43166/0 (socket says 172.21.15.72:43166) 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbb3400640 1 -- 172.21.15.72:0/311515870 learned_addr learned my addr 172.21.15.72:0/311515870 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.922+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76417580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 -- 172.21.15.72:0/311515870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 msgr2=0x559d76417580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76417580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 -- 172.21.15.72:0/311515870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 msgr2=0x559d7641c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 -- 172.21.15.72:0/311515870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d763208b0 con 0x559d7641d580 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbb3400640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642e920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0971d0 tx=0x7fbb9c098d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b4bd51491f76fda server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbba3e00640 1 -- 172.21.15.72:0/311515870 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c09a160 con 0x559d7641d580 2024-03-11T19:22:11.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbba3e00640 1 -- 172.21.15.72:0/311515870 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c0d3070 con 0x559d7641d580 2024-03-11T19:22:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.923+0000 7fbba3e00640 1 -- 172.21.15.72:0/311515870 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c09e950 con 0x559d7641d580 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 -- 172.21.15.72:0/311515870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 msgr2=0x559d7642e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642e920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0971d0 tx=0x7fbb9c098d10 comp rx=0 tx=0).stop 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 -- 172.21.15.72:0/311515870 shutdown_connections 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76417580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642e920 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 --2- 172.21.15.72:0/311515870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7641c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 -- 172.21.15.72:0/311515870 >> 172.21.15.72:0/311515870 conn(0x559d76414010 msgr2=0x559d7641bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 -- 172.21.15.72:0/311515870 shutdown_connections 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.924+0000 7fbbbd395740 1 -- 172.21.15.72:0/311515870 wait complete. 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 -- start start 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 Processor -- start 2024-03-11T19:22:11.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76492280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d7642c480 con 0x559d76418af0 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d764927c0 con 0x559d7641df70 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d76492930 con 0x559d7641d580 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbbd395740 10 client.0 _ll_register_callbacks cb 0x7fbb9c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbaaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43178/0 (socket says 172.21.15.72:43178) 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.925+0000 7fbbb3400640 1 -- 172.21.15.72:0/1641027264 learned_addr learned my addr 172.21.15.72:0/1641027264 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3e00640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76492280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 -- 172.21.15.72:0/1641027264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 msgr2=0x559d7642bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 -- 172.21.15.72:0/1641027264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 msgr2=0x559d76492280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76492280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 -- 172.21.15.72:0/1641027264 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d7634d4c0 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbb3400640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbba4011d10 tx=0x7fbba4009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729319d9e4c1c4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbba1600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40040b0 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fbba402ab00 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.926+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbba4004380 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba4004210 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559d76383ec0 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba4004380 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbba4005320 con 0x559d76418af0 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 10 client.15066 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-11T19:22:11.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 20 client.15066 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d764230e0 0x559d76494170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 4 client.15066 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 --> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d760d6270 con 0x559d764230e0 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.927+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d764230e0 0x559d76494170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbba1600640 10 client.15066 ms_handle_connect on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbaaa00640 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d764230e0 0x559d76494170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3c4f6fd2155914f server_cookie=80929940aece9d92 in_seq=0 out_seq=0 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbba1600640 1 -- 172.21.15.72:0/1641027264 <== mds.1 v2:172.21.15.72:6838/3478523060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x559d760d6270 con 0x559d764230e0 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbba1600640 10 client.15066 handle_command_reply: tid=0 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 client.15066 shutdown 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 msgr2=0x559d7642b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbba4011d10 tx=0x7fbba4009fa0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 shutdown_connections 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x559d764230e0 0x559d76494170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d7641df70 0x559d76492280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d7641d580 0x559d7642bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 --2- 172.21.15.72:0/1641027264 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d76418af0 0x559d7642b840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbba1600640 0 client.15066 ms_handle_reset on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.928+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 >> 172.21.15.72:0/1641027264 conn(0x559d76414010 msgr2=0x559d7641c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 shutdown_connections 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbbd395740 1 -- 172.21.15.72:0/1641027264 wait complete. 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbbd395740 20 client.15066 trim_cache size 0 max 16384 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24542, 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:11.937 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 118.712115146, 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 162.8724086 2024-03-11T19:22:11.938 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 msgr2=0x7fbb980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 0x7fbb980a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02c640 tx=0x7fbbac0326d0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 msgr2=0x7fbbb4199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00ea60 tx=0x7fbb9c00ef20 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 shutdown_connections 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fbb980a1a30 0x7fbb980a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb4104f70 0x7fbbb41a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbbb41045b0 0x7fbbb4199920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 --2- 172.21.15.72:0/525384907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb41033c0 0x7fbbb41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 >> 172.21.15.72:0/525384907 conn(0x7fbbb40fe930 msgr2=0x7fbbb41078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.929+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 shutdown_connections 2024-03-11T19:22:11.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:11.930+0000 7fbbba600640 1 -- 172.21.15.72:0/525384907 wait complete. 2024-03-11T19:22:11.949 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:22:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 -- 172.21.15.72:0/3235391921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e41077a0 msgr2=0x7fa3e4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3235391921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e41077a0 0x7fa3e4107bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009900 tx=0x7fa3d402bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:12.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 -- 172.21.15.72:0/3235391921 shutdown_connections 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3235391921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4109350 0x7fa3e410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3235391921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e4108990 0x7fa3e4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3235391921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e41077a0 0x7fa3e4107bc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 -- 172.21.15.72:0/3235391921 >> 172.21.15.72:0/3235391921 conn(0x7fa3e4076740 msgr2=0x7fa3e4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 -- 172.21.15.72:0/3235391921 shutdown_connections 2024-03-11T19:22:12.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.136+0000 7fa3eb400640 1 -- 172.21.15.72:0/3235391921 wait complete. 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 Processor -- start 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 -- start start 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 0x7fa3e419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e4109350 0x7fa3e41a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41a5350 con 0x7fa3e4108990 2024-03-11T19:22:12.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41a54c0 con 0x7fa3e4109350 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.137+0000 7fa3eb400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41a5680 con 0x7fa3e41077a0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 0x7fa3e419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43190/0 (socket says 172.21.15.72:43190) 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3dbe00640 1 -- 172.21.15.72:0/3923771796 learned_addr learned my addr 172.21.15.72:0/3923771796 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e9400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e4109350 0x7fa3e41a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 -- 172.21.15.72:0/3923771796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e4109350 msgr2=0x7fa3e41a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e4109350 0x7fa3e41a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 -- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 msgr2=0x7fa3e419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 -- 172.21.15.72:0/3923771796 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009560 con 0x7fa3e41077a0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.138+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 0x7fa3e419d860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4036040 tx=0x7fa3d4005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.139+0000 7fa3e2000640 1 -- 172.21.15.72:0/3923771796 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40043d0 con 0x7fa3e41077a0 2024-03-11T19:22:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.139+0000 7fa3e2000640 1 -- 172.21.15.72:0/3923771796 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3d40052d0 con 0x7fa3e41077a0 2024-03-11T19:22:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.139+0000 7fa3e2000640 1 -- 172.21.15.72:0/3923771796 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003cd0 con 0x7fa3e41077a0 2024-03-11T19:22:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.139+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e41a5900 con 0x7fa3e41077a0 2024-03-11T19:22:12.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.139+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e41a5d90 con 0x7fa3e41077a0 2024-03-11T19:22:12.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 Processor -- start 2024-03-11T19:22:12.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3e2000640 1 -- 172.21.15.72:0/3923771796 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa3d4005440 con 0x7fa3e41077a0 2024-03-11T19:22:12.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 -- start start 2024-03-11T19:22:12.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 0x5575905eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905ee0e0 0x5575905f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x557590557b30 con 0x5575905e8580 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x557590555060 con 0x5575905e8f70 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.142+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557590550840 con 0x5575905ee0e0 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43198/0 (socket says 172.21.15.72:43198) 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 -- 172.21.15.72:0/3373540112 learned_addr learned my addr 172.21.15.72:0/3373540112 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905ee0e0 0x5575905f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 0x5575905eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 -- 172.21.15.72:0/3373540112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905ee0e0 msgr2=0x5575905f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905ee0e0 0x5575905f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 -- 172.21.15.72:0/3373540112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 msgr2=0x5575905eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 0x5575905eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 -- 172.21.15.72:0/3373540112 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575905184c0 con 0x5575905e8580 2024-03-11T19:22:12.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 0x5575905eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:12.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e9400640 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009fc0 tx=0x7fa3dc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cd6d1c1839a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.143+0000 7fa3e2000640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 0x7fa3c80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.144+0000 7fa3e2000640 1 -- 172.21.15.72:0/3923771796 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa3d40ef900 con 0x7fa3e41077a0 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.144+0000 7fa3d9c00640 1 -- 172.21.15.72:0/3373540112 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc003b70 con 0x5575905e8580 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.144+0000 7fa3d9c00640 1 -- 172.21.15.72:0/3373540112 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3dc015070 con 0x5575905e8580 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.144+0000 7fa3d9c00640 1 -- 172.21.15.72:0/3373540112 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0118c0 con 0x5575905e8580 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.144+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 0x7fa3c80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/3373540112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 msgr2=0x5575905e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3e8a00640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 0x7fa3c80a3f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4006010 tx=0x7fa3d40b8040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009fc0 tx=0x7fa3dc010860 comp rx=0 tx=0).stop 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/3373540112 shutdown_connections 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905ee0e0 0x5575905f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8f70 0x5575905eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/3373540112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8580 0x5575905e89a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/3373540112 >> 172.21.15.72:0/3373540112 conn(0x5575905e3af0 msgr2=0x5575905e3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/3373540112 shutdown_connections 2024-03-11T19:22:12.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/3373540112 wait complete. 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 -- start start 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.145+0000 7fa3ee1ac740 1 Processor -- start 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575904f5ea0 con 0x5575905e8f70 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575904f5b40 con 0x5575905ee0e0 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557590298f20 con 0x5575905e8580 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3ee1ac740 10 client.0 _ll_register_callbacks cb 0x557590509950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43212/0 (socket says 172.21.15.72:43212) 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 -- 172.21.15.72:0/2621617766 learned_addr learned my addr 172.21.15.72:0/2621617766 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:12.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3e9400640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 -- 172.21.15.72:0/2621617766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 msgr2=0x5575905f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 -- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 msgr2=0x5575905f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.146+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3dbe00640 1 -- 172.21.15.72:0/2621617766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc009c70 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3e9400640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc012d20 tx=0x7fa3cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc80eedb17a8729 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa3cc012110 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557590574c40 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3cc00fe80 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3ee1ac740 10 client.15078 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.147+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55759054eec0 con 0x5575905e8f70 2024-03-11T19:22:12.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557590574c40 con 0x5575905e8f70 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa3cc010430 con 0x5575905e8f70 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 10 client.15078 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 20 client.15078 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5575905feae0 0x5575905fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 4 client.15078 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575902506c0 con 0x5575905feae0 2024-03-11T19:22:12.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.148+0000 7fa3e9400640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5575905feae0 0x5575905fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.149+0000 7fa3e9400640 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5575905feae0 0x5575905fdd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8911cc2dbe3e58 server_cookie=ad414324fa3c169a in_seq=0 out_seq=0 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.149+0000 7fa3d2000640 10 client.15078 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3d2000640 1 -- 172.21.15.72:0/2621617766 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575902506c0 con 0x5575905feae0 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3d2000640 10 client.15078 handle_command_reply: tid=0 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 client.15078 shutdown 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 msgr2=0x5575905f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc012d20 tx=0x7fa3cc00f790 comp rx=0 tx=0).stop 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 shutdown_connections 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5575905feae0 0x5575905fdd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f2950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/2621617766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3d2000640 0 client.15078 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 >> 172.21.15.72:0/2621617766 conn(0x5575905e3af0 msgr2=0x5575905e2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 shutdown_connections 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/2621617766 wait complete. 2024-03-11T19:22:12.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.150+0000 7fa3ee1ac740 20 client.15078 trim_cache size 0 max 16384 2024-03-11T19:22:12.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 Processor -- start 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 -- start start 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575905e8140 con 0x5575905e8f70 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575905e45e0 con 0x5575905ee0e0 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.168+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575905f0c30 con 0x5575905e8580 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36280/0 (socket says 172.21.15.72:36280) 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3e9400640 1 -- 172.21.15.72:0/466049679 learned_addr learned my addr 172.21.15.72:0/466049679 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:12.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3e9400640 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 -- 172.21.15.72:0/466049679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 msgr2=0x5575905e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 -- 172.21.15.72:0/466049679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 msgr2=0x5575905e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 -- 172.21.15.72:0/466049679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575904eb8b0 con 0x5575905e8f70 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f9950 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0107e0 tx=0x7fa3cc012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356e394ed9518b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.169+0000 7fa3d9200640 1 -- 172.21.15.72:0/466049679 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017040 con 0x5575905e8f70 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3d9200640 1 -- 172.21.15.72:0/466049679 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc00fe80 con 0x5575905e8f70 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3d9200640 1 -- 172.21.15.72:0/466049679 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0052d0 con 0x5575905e8f70 2024-03-11T19:22:12.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/466049679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 msgr2=0x5575905f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f9950 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0107e0 tx=0x7fa3cc012910 comp rx=0 tx=0).stop 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/466049679 shutdown_connections 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905ee0e0 0x5575905e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905e8f70 0x5575905f9950 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/466049679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8580 0x5575905e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/466049679 >> 172.21.15.72:0/466049679 conn(0x5575905e3af0 msgr2=0x5575905e3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/466049679 shutdown_connections 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.170+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/466049679 wait complete. 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 -- start start 2024-03-11T19:22:12.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 Processor -- start 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8580 0x5575905f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 0x5575906546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575905f9b00 con 0x5575905ee0e0 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575905f9c70 con 0x5575905e8580 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x557590654c20 con 0x5575905e8f70 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.171+0000 7fa3ee1ac740 10 client.0 _ll_register_callbacks cb 0x7fa3cc009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36294/0 (socket says 172.21.15.72:36294) 2024-03-11T19:22:12.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3e9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 0x5575906546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 -- 172.21.15.72:0/1273180453 learned_addr learned my addr 172.21.15.72:0/1273180453 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3e9400640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8580 0x5575905f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 -- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8580 msgr2=0x5575905f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8580 0x5575905f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 -- 172.21.15.72:0/1273180453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 msgr2=0x5575906546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 0x5575906546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 -- 172.21.15.72:0/1273180453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55759054eec0 con 0x5575905e8f70 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3e9400640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 0x5575906546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3dbe00640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc012cf0 tx=0x7fa3cc005f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c034917e75b1962 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.172+0000 7fa3d2000640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc003b40 con 0x5575905e8f70 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557590600170 con 0x5575905e8f70 2024-03-11T19:22:12.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa3cc000e40 con 0x5575905e8f70 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3cc005070 con 0x5575905e8f70 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 10 client.25189 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557590511510 con 0x5575905e8f70 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557590600170 con 0x5575905e8f70 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa3cc030070 con 0x5575905e8f70 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 10 client.25189 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 10 client.25189 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 20 client.25189 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.173+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x557590659d80 0x557590656200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3ee1ac740 4 client.25189 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa3dc015070 con 0x557590659d80 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3e9400640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x557590659d80 0x557590656200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3e9400640 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x557590659d80 0x557590656200 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7274601e22d4e051 server_cookie=daaa095d37f4fe50 in_seq=0 out_seq=0 2024-03-11T19:22:12.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3d2000640 10 client.25189 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3d2000640 1 -- 172.21.15.72:0/1273180453 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fa3dc015070 con 0x557590659d80 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3d2000640 10 client.25189 handle_command_reply: tid=0 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.174+0000 7fa3ee1ac740 1 client.25189 shutdown 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 msgr2=0x5575905f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc012cf0 tx=0x7fa3cc005f50 comp rx=0 tx=0).stop 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 shutdown_connections 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x557590659d80 0x557590656200 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5575905ee0e0 0x5575906546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3d2000640 0 client.25189 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5575905e8f70 0x5575905f6bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 --2- 172.21.15.72:0/1273180453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575905e8580 0x5575905f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 >> 172.21.15.72:0/1273180453 conn(0x5575905e3af0 msgr2=0x5575905e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 shutdown_connections 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 1 -- 172.21.15.72:0/1273180453 wait complete. 2024-03-11T19:22:12.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.175+0000 7fa3ee1ac740 20 client.25189 trim_cache size 0 max 16384 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24572, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 149.42326937999999, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:12.184 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 155.91399265999999 2024-03-11T19:22:12.185 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:12.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 msgr2=0x7fa3c80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 0x7fa3c80a3f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4006010 tx=0x7fa3d40b8040 comp rx=0 tx=0).stop 2024-03-11T19:22:12.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 msgr2=0x7fa3e419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:12.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 0x7fa3e419d860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4036040 tx=0x7fa3d4005e50 comp rx=0 tx=0).stop 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 shutdown_connections 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa3c80a1ab0 0x7fa3c80a3f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3e4109350 0x7fa3e41a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3e4108990 0x7fa3e419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 --2- 172.21.15.72:0/3923771796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3e41077a0 0x7fa3e419d860 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 >> 172.21.15.72:0/3923771796 conn(0x7fa3e4076740 msgr2=0x7fa3e410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 shutdown_connections 2024-03-11T19:22:12.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:12.176+0000 7fa3eb400640 1 -- 172.21.15.72:0/3923771796 wait complete. 2024-03-11T19:22:12.196 INFO:tasks.cephadm:Setup complete, yielding 2024-03-11T19:22:12.196 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:22:12.206 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:22:12.206 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-11T19:22:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:12 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/543051088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:12.661 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:12.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:12 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/543051088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:12 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/543051088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:13.502 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:13 smithi072 ceph-mon[31695]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:13.591 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:13 smithi143 ceph-mon[36013]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:13 smithi106 ceph-mon[35889]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 -- 172.21.15.72:0/3275617807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94103e80 msgr2=0x7f9a941042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 --2- 172.21.15.72:0/3275617807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94103e80 0x7f9a941042a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009990 tx=0x7f9a88009ca0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 -- 172.21.15.72:0/3275617807 shutdown_connections 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 --2- 172.21.15.72:0/3275617807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94105a30 0x7f9a9410c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 --2- 172.21.15.72:0/3275617807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94103e80 0x7f9a941042a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 --2- 172.21.15.72:0/3275617807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a941054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 -- 172.21.15.72:0/3275617807 >> 172.21.15.72:0/3275617807 conn(0x7f9a940ff490 msgr2=0x7f9a941018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:15.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 -- 172.21.15.72:0/3275617807 shutdown_connections 2024-03-11T19:22:15.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.295+0000 7f9a9c600700 1 -- 172.21.15.72:0/3275617807 wait complete. 2024-03-11T19:22:15.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.297+0000 7f9a9c600700 1 Processor -- start 2024-03-11T19:22:15.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.297+0000 7f9a9c600700 1 -- start start 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.297+0000 7f9a9c600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 0x7f9a94199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.297+0000 7f9a9c600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a9419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9c600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 0x7f9a941a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9c600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941a1a40 con 0x7f9a94105a30 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9c600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941a1b80 con 0x7f9a94103e80 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9c600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941a1d10 con 0x7f9a94105070 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 0x7f9a94199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 0x7f9a94199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36008/0 (socket says 172.21.15.72:36008) 2024-03-11T19:22:15.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99c00700 1 -- 172.21.15.72:0/4280060510 learned_addr learned my addr 172.21.15.72:0/4280060510 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a9419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9a600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 0x7f9a941a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 -- 172.21.15.72:0/4280060510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 msgr2=0x7f9a94199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 0x7f9a94199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 -- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 msgr2=0x7f9a941a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 0x7f9a941a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a99200700 1 -- 172.21.15.72:0/4280060510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880095f0 con 0x7f9a94105070 2024-03-11T19:22:15.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.298+0000 7f9a9a600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 0x7f9a941a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:15.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.299+0000 7f9a99200700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a9419a3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e9f0 tx=0x7f9a8000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:15.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.302+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000eee0 con 0x7f9a94105070 2024-03-11T19:22:15.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.302+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a80004500 con 0x7f9a94105070 2024-03-11T19:22:15.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.302+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80003df0 con 0x7f9a94105070 2024-03-11T19:22:15.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.302+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a941a1ff0 con 0x7f9a94105070 2024-03-11T19:22:15.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.302+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a941a24b0 con 0x7f9a94105070 2024-03-11T19:22:15.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.303+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a94068930 con 0x7f9a94105070 2024-03-11T19:22:15.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.305+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9a80004020 con 0x7f9a94105070 2024-03-11T19:22:15.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.306+0000 7f9a92000700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 0x7f9a7c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:15.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.306+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f9a80014070 con 0x7f9a94105070 2024-03-11T19:22:15.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.306+0000 7f9a99c00700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 0x7f9a7c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:15.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.307+0000 7f9a99c00700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 0x7f9a7c0a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800b7c0 tx=0x7f9a88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:15.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.310+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a80096800 con 0x7f9a94105070 2024-03-11T19:22:15.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:15 smithi072 ceph-mon[31695]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:15.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.550+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a9402d910 con 0x7f9a7c0a1b40 2024-03-11T19:22:15.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.552+0000 7f9a92000700 1 -- 172.21.15.72:0/4280060510 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9a9402d910 con 0x7f9a7c0a1b40 2024-03-11T19:22:15.556 INFO:teuthology.orchestra.run.smithi072.stdout:Backend: cephadm 2024-03-11T19:22:15.557 INFO:teuthology.orchestra.run.smithi072.stdout:Available: Yes 2024-03-11T19:22:15.557 INFO:teuthology.orchestra.run.smithi072.stdout:Paused: No 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 msgr2=0x7f9a7c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 0x7f9a7c0a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800b7c0 tx=0x7f9a88005fb0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 msgr2=0x7f9a9419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a9419a3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e9f0 tx=0x7f9a8000ed00 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 shutdown_connections 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f9a7c0a1b40 0x7f9a7c0a4000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a94105a30 0x7f9a941a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a94103e80 0x7f9a94199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 --2- 172.21.15.72:0/4280060510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a94105070 0x7f9a9419a3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 >> 172.21.15.72:0/4280060510 conn(0x7f9a940ff490 msgr2=0x7f9a94108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.555+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 shutdown_connections 2024-03-11T19:22:15.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:15.556+0000 7f9a9c600700 1 -- 172.21.15.72:0/4280060510 wait complete. 2024-03-11T19:22:15.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:15 smithi106 ceph-mon[35889]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:15.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:15 smithi143 ceph-mon[36013]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:16.712 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-11T19:22:17.175 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:17.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:17 smithi072 ceph-mon[31695]: from='client.25195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:17.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:17 smithi072 ceph-mon[31695]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:17 smithi106 ceph-mon[35889]: from='client.25195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:17.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:17 smithi106 ceph-mon[35889]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:17.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:17 smithi143 ceph-mon[36013]: from='client.25195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:17.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:17 smithi143 ceph-mon[36013]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:19.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:19 smithi072 ceph-mon[31695]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:19.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:19 smithi106 ceph-mon[35889]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:19.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:19 smithi143 ceph-mon[36013]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:20.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 -- 172.21.15.72:0/1413200889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c078230 msgr2=0x7f4a8c078ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:20.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 --2- 172.21.15.72:0/1413200889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c078230 0x7f4a8c078ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009f90 tx=0x7f4a8400b330 comp rx=0 tx=0).stop 2024-03-11T19:22:20.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 -- 172.21.15.72:0/1413200889 shutdown_connections 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 --2- 172.21.15.72:0/1413200889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c078230 0x7f4a8c078ab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 --2- 172.21.15.72:0/1413200889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c076620 0x7f4a8c076a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 --2- 172.21.15.72:0/1413200889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 0x7f4a8c077cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 -- 172.21.15.72:0/1413200889 >> 172.21.15.72:0/1413200889 conn(0x7f4a8c0ff490 msgr2=0x7f4a8c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 -- 172.21.15.72:0/1413200889 shutdown_connections 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.023+0000 7f4a91600700 1 -- 172.21.15.72:0/1413200889 wait complete. 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 Processor -- start 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 -- start start 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:20.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 0x7f4a8c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c078230 0x7f4a8c1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1a5ed0 con 0x7f4a8c078230 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1a6010 con 0x7f4a8c076620 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.024+0000 7f4a91600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1a61a0 con 0x7f4a8c077870 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36030/0 (socket says 172.21.15.72:36030) 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 -- 172.21.15.72:0/1653380159 learned_addr learned my addr 172.21.15.72:0/1653380159 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a83e00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 0x7f4a8c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8b400700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c078230 0x7f4a8c1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 -- 172.21.15.72:0/1653380159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 msgr2=0x7f4a8c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 0x7f4a8c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 -- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c078230 msgr2=0x7f4a8c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c078230 0x7f4a8c1a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.025+0000 7f4a8aa00700 1 -- 172.21.15.72:0/1653380159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a84009c40 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a8aa00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00e9c0 tx=0x7f4a7c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c004a30 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c1a6480 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c00cca0 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c005580 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.026+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c1a69d0 con 0x7f4a8c076620 2024-03-11T19:22:20.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.027+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c1108e0 con 0x7f4a8c076620 2024-03-11T19:22:20.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.029+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4a7c004c20 con 0x7f4a8c076620 2024-03-11T19:22:20.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.030+0000 7f4a83400700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 0x7f4a700a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:20.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.031+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f4a7c014070 con 0x7f4a8c076620 2024-03-11T19:22:20.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.031+0000 7f4a83e00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 0x7f4a700a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:20.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.031+0000 7f4a83e00700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 0x7f4a700a4050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009c60 tx=0x7f4a74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:20.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.035+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a7c096630 con 0x7f4a8c076620 2024-03-11T19:22:20.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.264+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a8c02d8a0 con 0x7f4a700a1b90 2024-03-11T19:22:20.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.305+0000 7f4a83400700 1 -- 172.21.15.72:0/1653380159 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f4a8c02d8a0 con 0x7f4a700a1b90 2024-03-11T19:22:20.310 INFO:teuthology.orchestra.run.smithi072.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-11T19:22:20.310 INFO:teuthology.orchestra.run.smithi072.stdout:mds.a smithi072 running (2m) 31s ago 2m 12.4M - 19.0.0-1607-g94a53970 a6477a474d71 9f551f20746b 2024-03-11T19:22:20.310 INFO:teuthology.orchestra.run.smithi072.stdout:mds.b smithi106 running (2m) 95s ago 2m 25.6M - 19.0.0-1607-g94a53970 a6477a474d71 54a3ab9fb018 2024-03-11T19:22:20.310 INFO:teuthology.orchestra.run.smithi072.stdout:mds.c smithi143 running (3m) 95s ago 3m 15.7M - 19.0.0-1607-g94a53970 a6477a474d71 5d919a875e69 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.d smithi072 running (3m) 31s ago 3m 12.5M - 19.0.0-1607-g94a53970 a6477a474d71 dfa2c00646d3 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.e smithi106 running (3m) 95s ago 3m 13.0M - 19.0.0-1607-g94a53970 a6477a474d71 ab62c213db27 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.f smithi143 running (3m) 95s ago 3m 14.7M - 19.0.0-1607-g94a53970 a6477a474d71 bc41a8c6d729 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.g smithi072 running (2m) 31s ago 2m 16.4M - 19.0.0-1607-g94a53970 a6477a474d71 a38a0270e103 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.h smithi106 running (2m) 95s ago 2m 18.2M - 19.0.0-1607-g94a53970 a6477a474d71 344166ecaf73 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.i smithi143 running (2m) 95s ago 2m 16.9M - 19.0.0-1607-g94a53970 a6477a474d71 b6d009ae86ad 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.j smithi072 running (3m) 31s ago 3m 12.5M - 19.0.0-1607-g94a53970 a6477a474d71 7973b143821e 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.k smithi106 running (3m) 95s ago 3m 14.1M - 19.0.0-1607-g94a53970 a6477a474d71 faa348744079 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mds.l smithi143 running (3m) 95s ago 3m 14.0M - 19.0.0-1607-g94a53970 a6477a474d71 9197895d804a 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.x smithi072 *:9283,8765 running (11m) 31s ago 11m 514M - 19.0.0-1607-g94a53970 a6477a474d71 896de6c503c8 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.y smithi106 *:8443,8765 running (9m) 95s ago 9m 451M - 19.0.0-1607-g94a53970 a6477a474d71 3596117ded4d 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.z smithi143 *:8443,8765 running (9m) 95s ago 9m 452M - 19.0.0-1607-g94a53970 a6477a474d71 90aed54f4f60 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mon.a smithi072 running (11m) 31s ago 11m 48.5M 2048M 19.0.0-1607-g94a53970 a6477a474d71 d1b9521f6196 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mon.b smithi106 running (9m) 95s ago 9m 41.1M 2048M 19.0.0-1607-g94a53970 a6477a474d71 57037ee4fb7e 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:mon.c smithi143 running (9m) 95s ago 9m 41.4M 2048M 19.0.0-1607-g94a53970 a6477a474d71 406f788138df 2024-03-11T19:22:20.311 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 smithi072 running (8m) 31s ago 8m 44.9M 4272M 19.0.0-1607-g94a53970 a6477a474d71 99ac7e45d256 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 smithi106 running (8m) 95s ago 8m 59.6M 4270M 19.0.0-1607-g94a53970 a6477a474d71 8ffb48423c34 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 smithi143 running (8m) 95s ago 8m 57.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 60f7effeec28 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 smithi072 running (7m) 31s ago 7m 54.2M 4272M 19.0.0-1607-g94a53970 a6477a474d71 9fe7d2b3957b 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 smithi106 running (7m) 95s ago 7m 52.9M 4270M 19.0.0-1607-g94a53970 a6477a474d71 b29cf4e67508 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 smithi143 running (6m) 95s ago 6m 57.0M 4270M 19.0.0-1607-g94a53970 a6477a474d71 c6012eb15228 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 smithi072 running (6m) 31s ago 6m 53.5M 4272M 19.0.0-1607-g94a53970 a6477a474d71 ff8250d80362 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 smithi106 running (5m) 95s ago 5m 57.6M 4270M 19.0.0-1607-g94a53970 a6477a474d71 4e60e5997e01 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 smithi143 running (5m) 95s ago 5m 43.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 e383f6f3bb51 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 smithi072 running (4m) 31s ago 4m 44.3M 4272M 19.0.0-1607-g94a53970 a6477a474d71 6a1ad1cf7a70 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 smithi106 running (4m) 95s ago 4m 43.3M 4270M 19.0.0-1607-g94a53970 a6477a474d71 19492282dd0c 2024-03-11T19:22:20.312 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 smithi143 running (3m) 95s ago 3m 52.8M 4270M 19.0.0-1607-g94a53970 a6477a474d71 e63421d357bb 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 msgr2=0x7f4a700a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 0x7f4a700a4050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009c60 tx=0x7f4a74009260 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 msgr2=0x7f4a8c19e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00e9c0 tx=0x7f4a7c00ecd0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 shutdown_connections 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f4a700a1b90 0x7f4a700a4050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a8c078230 0x7f4a8c1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a8c076620 0x7f4a8c19e340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 --2- 172.21.15.72:0/1653380159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a8c077870 0x7f4a8c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.308+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 >> 172.21.15.72:0/1653380159 conn(0x7f4a8c0ff490 msgr2=0x7f4a8c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:20.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.309+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 shutdown_connections 2024-03-11T19:22:20.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:20.309+0000 7f4a91600700 1 -- 172.21.15.72:0/1653380159 wait complete. 2024-03-11T19:22:21.420 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-11T19:22:21.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:21 smithi106 ceph-mon[35889]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:21.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:21 smithi106 ceph-mon[35889]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:21 smithi072 ceph-mon[31695]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:21.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:21 smithi072 ceph-mon[31695]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:21 smithi143 ceph-mon[36013]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:21.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:21 smithi143 ceph-mon[36013]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:21.854 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:23.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:23 smithi106 ceph-mon[35889]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:23.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:23 smithi072 ceph-mon[31695]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:23.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:23 smithi143 ceph-mon[36013]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:24.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- 172.21.15.72:0/3063867056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 msgr2=0x7fca84074100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 --2- 172.21.15.72:0/3063867056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca84074100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca74009990 tx=0x7fca74009ca0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- 172.21.15.72:0/3063867056 shutdown_connections 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 --2- 172.21.15.72:0/3063867056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca8410c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 --2- 172.21.15.72:0/3063867056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca84074100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 --2- 172.21.15.72:0/3063867056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 0x7fca84074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- 172.21.15.72:0/3063867056 >> 172.21.15.72:0/3063867056 conn(0x7fca840fd210 msgr2=0x7fca840ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- 172.21.15.72:0/3063867056 shutdown_connections 2024-03-11T19:22:24.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- 172.21.15.72:0/3063867056 wait complete. 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 Processor -- start 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.403+0000 7fca88c00700 1 -- start start 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 0x7fca84199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca841a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca841a1a50 con 0x7fca84109b80 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca841a1b90 con 0x7fca840746d0 2024-03-11T19:22:24.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca88c00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca841a1d20 con 0x7fca84075ca0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca81400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca81400700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:54478/0 (socket says 172.21.15.72:54478) 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca82800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca841a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca81400700 1 -- 172.21.15.72:0/3173972204 learned_addr learned my addr 172.21.15.72:0/3173972204 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.404+0000 7fca81e00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 0x7fca84199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 -- 172.21.15.72:0/3173972204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 msgr2=0x7fca84199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 0x7fca84199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 -- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 msgr2=0x7fca841a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca841a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 -- 172.21.15.72:0/3173972204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740095f0 con 0x7fca84075ca0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca82800700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca841a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca81400700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca7000e9a0 tx=0x7fca7000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:24.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70009790 con 0x7fca84075ca0 2024-03-11T19:22:24.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca7000ccf0 con 0x7fca84075ca0 2024-03-11T19:22:24.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca841a1fa0 con 0x7fca84075ca0 2024-03-11T19:22:24.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70003d20 con 0x7fca84075ca0 2024-03-11T19:22:24.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.405+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca841a2470 con 0x7fca84075ca0 2024-03-11T19:22:24.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.406+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca84068930 con 0x7fca84075ca0 2024-03-11T19:22:24.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.408+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fca70004390 con 0x7fca84075ca0 2024-03-11T19:22:24.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.411+0000 7fca7aa00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 0x7fca680a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:24.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.411+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fca70014070 con 0x7fca84075ca0 2024-03-11T19:22:24.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.411+0000 7fca81e00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 0x7fca680a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:24.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.411+0000 7fca81e00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 0x7fca680a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fca7400b6f0 tx=0x7fca74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:24.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.415+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca70096810 con 0x7fca84075ca0 2024-03-11T19:22:24.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:24 smithi106 ceph-mon[35889]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:24.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.644+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fca8402d910 con 0x7fca680a1ac0 2024-03-11T19:22:24.647 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:24 smithi072 ceph-mon[31695]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:24.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.659+0000 7fca7aa00700 1 -- 172.21.15.72:0/3173972204 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fca8402d910 con 0x7fca680a1ac0 2024-03-11T19:22:24.663 INFO:teuthology.orchestra.run.smithi072.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-11T19:22:24.663 INFO:teuthology.orchestra.run.smithi072.stdout:mds.all 12/12 100s ago 3m smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-03-11T19:22:24.663 INFO:teuthology.orchestra.run.smithi072.stdout:mgr 3/3 100s ago 9m smithi072=x;smithi106=y;smithi143=z;count:3 2024-03-11T19:22:24.663 INFO:teuthology.orchestra.run.smithi072.stdout:mon 3/3 100s ago 10m smithi072:172.21.15.72=a;smithi106:172.21.15.106=b;smithi143:172.21.15.143=c;count:3 2024-03-11T19:22:24.664 INFO:teuthology.orchestra.run.smithi072.stdout:osd 12 100s ago - 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 msgr2=0x7fca680a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 0x7fca680a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fca7400b6f0 tx=0x7fca74005fb0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 msgr2=0x7fca8419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca7000e9a0 tx=0x7fca7000ecb0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 shutdown_connections 2024-03-11T19:22:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fca680a1ac0 0x7fca680a3f80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca84109b80 0x7fca841a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca840746d0 0x7fca84199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 --2- 172.21.15.72:0/3173972204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca84075ca0 0x7fca8419a400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 >> 172.21.15.72:0/3173972204 conn(0x7fca840fd210 msgr2=0x7fca84105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 shutdown_connections 2024-03-11T19:22:24.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:24.662+0000 7fca88c00700 1 -- 172.21.15.72:0/3173972204 wait complete. 2024-03-11T19:22:24.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:24 smithi143 ceph-mon[36013]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:25.637 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:25 smithi072 ceph-mon[31695]: from='client.25207 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:25.640 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-11T19:22:25.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:25 smithi106 ceph-mon[35889]: from='client.25207 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:25.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:25 smithi143 ceph-mon[36013]: from='client.25207 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:26.105 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:26.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:26 smithi106 ceph-mon[35889]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:26.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:26 smithi072 ceph-mon[31695]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:26.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:26 smithi143 ceph-mon[36013]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 -- 172.21.15.72:0/306861151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a010e2a0 msgr2=0x7f24a0110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 --2- 172.21.15.72:0/306861151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a010e2a0 0x7f24a0110690 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2498009f90 tx=0x7f249800b330 comp rx=0 tx=0).stop 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 -- 172.21.15.72:0/306861151 shutdown_connections 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 --2- 172.21.15.72:0/306861151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a010e2a0 0x7f24a0110690 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 --2- 172.21.15.72:0/306861151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a0075ca0 0x7f24a0074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 --2- 172.21.15.72:0/306861151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 0x7f24a0074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:28.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.809+0000 7f24a5200700 1 -- 172.21.15.72:0/306861151 >> 172.21.15.72:0/306861151 conn(0x7f24a00fd220 msgr2=0x7f24a00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:28.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.810+0000 7f24a5200700 1 -- 172.21.15.72:0/306861151 shutdown_connections 2024-03-11T19:22:28.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.810+0000 7f24a5200700 1 -- 172.21.15.72:0/306861151 wait complete. 2024-03-11T19:22:28.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 Processor -- start 2024-03-11T19:22:28.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 -- start start 2024-03-11T19:22:28.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 0x7f24a019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:28.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:28.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a010e2a0 0x7f24a01a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:28.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a5f50 con 0x7f24a010e2a0 2024-03-11T19:22:28.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a6090 con 0x7f24a0075ca0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.811+0000 7f24a5200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a6220 con 0x7f24a00746d0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249de00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 0x7f24a019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249e800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a010e2a0 0x7f24a01a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45468/0 (socket says 172.21.15.72:45468) 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 -- 172.21.15.72:0/772484517 learned_addr learned my addr 172.21.15.72:0/772484517 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 -- 172.21.15.72:0/772484517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 msgr2=0x7f24a019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 0x7f24a019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 -- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a010e2a0 msgr2=0x7f24a01a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a010e2a0 0x7f24a01a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.812+0000 7f249d400700 1 -- 172.21.15.72:0/772484517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009c40 con 0x7f24a0075ca0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f249d400700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e970 tx=0x7f248c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:28.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c009790 con 0x7f24a0075ca0 2024-03-11T19:22:28.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f248c00cca0 con 0x7f24a0075ca0 2024-03-11T19:22:28.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a01a6500 con 0x7f24a0075ca0 2024-03-11T19:22:28.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c003ce0 con 0x7f24a0075ca0 2024-03-11T19:22:28.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.813+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a01a6a50 con 0x7f24a0075ca0 2024-03-11T19:22:28.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.814+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a0104500 con 0x7f24a0075ca0 2024-03-11T19:22:28.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.816+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f248c00c7c0 con 0x7f24a0075ca0 2024-03-11T19:22:28.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.817+0000 7f2496a00700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 0x7f24840a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:28.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.817+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f248c014070 con 0x7f24a0075ca0 2024-03-11T19:22:28.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.818+0000 7f249de00700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 0x7f24840a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:28.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.818+0000 7f249de00700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 0x7f24840a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24a0076080 tx=0x7f2490009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:28.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:28.820+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f248c0967f0 con 0x7f24a0075ca0 2024-03-11T19:22:29.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.050+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f24a002d8d0 con 0x7f24840a1b40 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.053+0000 7f2496a00700 1 -- 172.21.15.72:0/772484517 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f24a002d8d0 con 0x7f24840a1b40 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stdout:HOST ADDR LABELS STATUS 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 172.21.15.72 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stdout:smithi106 172.21.15.106 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stdout:smithi143 172.21.15.143 2024-03-11T19:22:29.057 INFO:teuthology.orchestra.run.smithi072.stdout:3 hosts in cluster 2024-03-11T19:22:29.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 msgr2=0x7f24840a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:29.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 0x7f24840a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24a0076080 tx=0x7f2490009310 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 msgr2=0x7f24a019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e970 tx=0x7f248c00ec80 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 shutdown_connections 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f24840a1b40 0x7f24840a4000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f24a010e2a0 0x7f24a01a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24a00746d0 0x7f24a019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 --2- 172.21.15.72:0/772484517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24a0075ca0 0x7f24a019e950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 >> 172.21.15.72:0/772484517 conn(0x7f24a00fd220 msgr2=0x7f24a0101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 shutdown_connections 2024-03-11T19:22:29.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:29.057+0000 7f24a5200700 1 -- 172.21.15.72:0/772484517 wait complete. 2024-03-11T19:22:29.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:29 smithi072 ceph-mon[31695]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:29.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:29 smithi106 ceph-mon[35889]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:29.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:29 smithi143 ceph-mon[36013]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:30.086 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-11T19:22:30.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:30 smithi072 ceph-mon[31695]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:30.531 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:30.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:30 smithi106 ceph-mon[35889]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:30.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:30 smithi143 ceph-mon[36013]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:31.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:31 smithi072 ceph-mon[31695]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:31.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:31 smithi106 ceph-mon[35889]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:31.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:31 smithi143 ceph-mon[36013]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:33.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.051+0000 7f1a27200700 1 -- 172.21.15.72:0/2414656481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28104fc0 msgr2=0x7f1a28105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:33.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 --2- 172.21.15.72:0/2414656481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28104fc0 0x7f1a28105420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1a140099c0 tx=0x7f1a14009cd0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 -- 172.21.15.72:0/2414656481 shutdown_connections 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 --2- 172.21.15.72:0/2414656481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28105960 0x7f1a2810c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 --2- 172.21.15.72:0/2414656481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a281041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 --2- 172.21.15.72:0/2414656481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28104fc0 0x7f1a28105420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 -- 172.21.15.72:0/2414656481 >> 172.21.15.72:0/2414656481 conn(0x7f1a280ff360 msgr2=0x7f1a281017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 -- 172.21.15.72:0/2414656481 shutdown_connections 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.052+0000 7f1a27200700 1 -- 172.21.15.72:0/2414656481 wait complete. 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.053+0000 7f1a27200700 1 Processor -- start 2024-03-11T19:22:33.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.053+0000 7f1a27200700 1 -- start start 2024-03-11T19:22:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28104fc0 0x7f1a2819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:33.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28105960 0x7f1a281a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a1860 con 0x7f1a28103dc0 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a19a0 con 0x7f1a28105960 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a27200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a1b60 con 0x7f1a28104fc0 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a25e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a25e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33630/0 (socket says 172.21.15.72:33630) 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a25e00700 1 -- 172.21.15.72:0/3876464575 learned_addr learned my addr 172.21.15.72:0/3876464575 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a26800700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28105960 0x7f1a281a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:33.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.054+0000 7f1a25400700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28104fc0 0x7f1a2819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 -- 172.21.15.72:0/3876464575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28104fc0 msgr2=0x7f1a2819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28104fc0 0x7f1a2819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 -- 172.21.15.72:0/3876464575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28105960 msgr2=0x7f1a281a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28105960 0x7f1a281a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 -- 172.21.15.72:0/3876464575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140095f0 con 0x7f1a28103dc0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a25e00700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e970 tx=0x7f1a1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18009790 con 0x7f1a28103dc0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a1800ccc0 con 0x7f1a28103dc0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.055+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a281a1e40 con 0x7f1a28103dc0 2024-03-11T19:22:33.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.056+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18003d80 con 0x7f1a28103dc0 2024-03-11T19:22:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.056+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a281a2250 con 0x7f1a28103dc0 2024-03-11T19:22:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.057+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a28068930 con 0x7f1a28103dc0 2024-03-11T19:22:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.059+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1a1800c7e0 con 0x7f1a28103dc0 2024-03-11T19:22:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.060+0000 7f1a1ea00700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 0x7f1a0c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.060+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f1a18014070 con 0x7f1a28103dc0 2024-03-11T19:22:33.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.060+0000 7f1a25400700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 0x7f1a0c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:33.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.061+0000 7f1a25400700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 0x7f1a0c0a3f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a140036c0 tx=0x7f1a14005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:33.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.064+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a18096780 con 0x7f1a28103dc0 2024-03-11T19:22:33.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.295+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a2802d990 con 0x7f1a0c0a1ac0 2024-03-11T19:22:33.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.299+0000 7f1a1ea00700 1 -- 172.21.15.72:0/3876464575 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f1a2802d990 con 0x7f1a0c0a1ac0 2024-03-11T19:22:33.302 INFO:teuthology.orchestra.run.smithi072.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-11T19:22:33.302 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-11T19:22:33.303 INFO:teuthology.orchestra.run.smithi072.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-11T19:22:33.303 INFO:teuthology.orchestra.run.smithi072.stdout:smithi143 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330073400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-11T19:22:33.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 msgr2=0x7f1a0c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:33.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 0x7f1a0c0a3f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a140036c0 tx=0x7f1a14005bc0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 msgr2=0x7f1a28199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:33.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e970 tx=0x7f1a1800ec80 comp rx=0 tx=0).stop 2024-03-11T19:22:33.305 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:33 smithi072 ceph-mon[31695]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 shutdown_connections 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f1a0c0a1ac0 0x7f1a0c0a3f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a28105960 0x7f1a281a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a28103dc0 0x7f1a28199d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 --2- 172.21.15.72:0/3876464575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a28104fc0 0x7f1a2819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.302+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 >> 172.21.15.72:0/3876464575 conn(0x7f1a280ff360 msgr2=0x7f1a281082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.303+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 shutdown_connections 2024-03-11T19:22:33.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:33.303+0000 7f1a27200700 1 -- 172.21.15.72:0/3876464575 wait complete. 2024-03-11T19:22:33.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:33 smithi106 ceph-mon[35889]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:33.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:33 smithi143 ceph-mon[36013]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:34.437 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:34 smithi072 ceph-mon[31695]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:34.441 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:22:34.451 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:22:34.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-11T19:22:34.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:34 smithi106 ceph-mon[35889]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:34.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:34 smithi143 ceph-mon[36013]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:22:34.900 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:35.300 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:35 smithi072 ceph-mon[31695]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:35.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:35 smithi106 ceph-mon[35889]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:35.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:35 smithi143 ceph-mon[36013]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:37.492 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:37 smithi072 ceph-mon[31695]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 -- 172.21.15.72:0/193458126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0075c20 msgr2=0x7f04a0074080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 --2- 172.21.15.72:0/193458126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0075c20 0x7f04a0074080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f049000b0a0 tx=0x7f049000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 -- 172.21.15.72:0/193458126 shutdown_connections 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 --2- 172.21.15.72:0/193458126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 --2- 172.21.15.72:0/193458126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0075c20 0x7f04a0074080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 --2- 172.21.15.72:0/193458126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0074650 0x7f04a0074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 -- 172.21.15.72:0/193458126 >> 172.21.15.72:0/193458126 conn(0x7f04a00fd290 msgr2=0x7f04a00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 -- 172.21.15.72:0/193458126 shutdown_connections 2024-03-11T19:22:37.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.527+0000 7f04a4e00700 1 -- 172.21.15.72:0/193458126 wait complete. 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.528+0000 7f04a4e00700 1 Processor -- start 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.528+0000 7f04a4e00700 1 -- start start 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0075c20 0x7f04a019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a01a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a1b60 con 0x7f04a0074650 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a1ca0 con 0x7f04a0109bf0 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f04a4e00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a1e30 con 0x7f04a0075c20 2024-03-11T19:22:37.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33642/0 (socket says 172.21.15.72:33642) 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 -- 172.21.15.72:0/3608028425 learned_addr learned my addr 172.21.15.72:0/3608028425 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f0495400700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0075c20 0x7f04a019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049e800700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a01a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 -- 172.21.15.72:0/3608028425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0075c20 msgr2=0x7f04a019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0075c20 0x7f04a019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 -- 172.21.15.72:0/3608028425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 msgr2=0x7f04a01a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a01a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049de00700 1 -- 172.21.15.72:0/3608028425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490009d00 con 0x7f04a0074650 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.529+0000 7f049e800700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a01a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f049de00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04900079b0 tx=0x7f049000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049000e040 con 0x7f04a0074650 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0490004c50 con 0x7f04a0074650 2024-03-11T19:22:37.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a01a20b0 con 0x7f04a0074650 2024-03-11T19:22:37.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900052a0 con 0x7f04a0074650 2024-03-11T19:22:37.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.530+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a01a2520 con 0x7f04a0074650 2024-03-11T19:22:37.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.531+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a0108a00 con 0x7f04a0074650 2024-03-11T19:22:37.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.533+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f049001d470 con 0x7f04a0074650 2024-03-11T19:22:37.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.534+0000 7f0497400700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 0x7f04840a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:37.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.535+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f04900cec60 con 0x7f04a0074650 2024-03-11T19:22:37.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.535+0000 7f0495400700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 0x7f04840a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:37.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.535+0000 7f0495400700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 0x7f04840a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0488009c60 tx=0x7f0488009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:37.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.539+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0490097b20 con 0x7f04a0074650 2024-03-11T19:22:37.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:37 smithi106 ceph-mon[35889]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:37.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:37 smithi143 ceph-mon[36013]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:37.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.808+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f04a0068930 con 0x7f04a0074650 2024-03-11T19:22:37.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.809+0000 7f0497400700 1 -- 172.21.15.72:0/3608028425 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2945 (secure 0 0 0) 0x7f049009b1e0 con 0x7f04a0074650 2024-03-11T19:22:37.813 INFO:teuthology.orchestra.run.smithi072.stdout:e24 2024-03-11T19:22:37.813 INFO:teuthology.orchestra.run.smithi072.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:22:37.813 INFO:teuthology.orchestra.run.smithi072.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:22:37.813 INFO:teuthology.orchestra.run.smithi072.stdout:legacy client fscid: 1 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:Filesystem 'cephfs' (1) 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:fs_name cephfs 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 24 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-03-11T19:19:41.837094+0000 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-03-11T19:20:15.233431+0000 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:tableserver 0 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:root 0 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:session_timeout 300 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:session_autoclose 300 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:max_file_size 1099511627776 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:max_xattr_size 65536 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:required_client_features {} 2024-03-11T19:22:37.814 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure 0 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure_osd_epoch 0 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:max_mds 3 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:in 0,1,2 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:up {0=24572,1=24542,2=24532} 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:failed 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:damaged 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:stopped 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:data_pools [3,4] 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:metadata_pool 2 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:inline_data disabled 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:balancer 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:bal_rank_mask -1 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:standby_count_wanted 0 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b{0:24572} state up:active seq 3 addr [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.815 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.h{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.106:6836/2982797341,v1:172.21.15.106:6837/2982797341] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.g{1:24542} state up:active seq 13 addr [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c{1:24514} state up:standby-replay seq 1 addr [v2:172.21.15.143:6836/902566146,v1:172.21.15.143:6837/902566146] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.i{2:24532} state up:active seq 14 addr [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.f{2:24502} state up:standby-replay seq 1 addr [v2:172.21.15.143:6834/2743792791,v1:172.21.15.143:6835/2743792791] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:Standby daemons: 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.j{-1:14379} state up:standby seq 1 addr [v2:172.21.15.72:6834/1028801800,v1:172.21.15.72:6835/1028801800] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a{-1:14463} state up:standby seq 1 addr [v2:172.21.15.72:6840/1103846445,v1:172.21.15.72:6841/1103846445] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.106:6832/4123124925,v1:172.21.15.106:6833/4123124925] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.143:6832/239358886,v1:172.21.15.143:6833/239358886] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d{-1:24484} state up:standby seq 1 addr [v2:172.21.15.72:6836/1584863054,v1:172.21.15.72:6837/1584863054] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.816 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.e{-1:24488} state up:standby seq 1 addr [v2:172.21.15.106:6834/456413378,v1:172.21.15.106:6835/456413378] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.812+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 msgr2=0x7f04840a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.812+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 0x7f04840a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0488009c60 tx=0x7f0488009260 comp rx=0 tx=0).stop 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.812+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 msgr2=0x7f04a0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.812+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04900079b0 tx=0x7f049000bab0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 shutdown_connections 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f04840a1ac0 0x7f04840a3f80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04a0109bf0 0x7f04a01a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04a0074650 0x7f04a0199fd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 --2- 172.21.15.72:0/3608028425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04a0075c20 0x7f04a019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:37.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 >> 172.21.15.72:0/3608028425 conn(0x7f04a00fd290 msgr2=0x7f04a0105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:37.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 shutdown_connections 2024-03-11T19:22:37.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:37.813+0000 7f04a4e00700 1 -- 172.21.15.72:0/3608028425 wait complete. 2024-03-11T19:22:37.818 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:38.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:38 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3608028425' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:22:38.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:38 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3608028425' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:22:38.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:38 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3608028425' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:22:38.974 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-11T19:22:39.427 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:39.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:39 smithi072 ceph-mon[31695]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:39.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:39 smithi106 ceph-mon[35889]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:39.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:39 smithi143 ceph-mon[36013]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:41.491 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:41 smithi072 ceph-mon[31695]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:41.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:41 smithi106 ceph-mon[35889]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:41.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:41 smithi143 ceph-mon[36013]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 -- 172.21.15.72:0/708626178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c107a50 msgr2=0x7f763c109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 --2- 172.21.15.72:0/708626178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c107a50 0x7f763c109e40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f763400b0a0 tx=0x7f763400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 -- 172.21.15.72:0/708626178 shutdown_connections 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 --2- 172.21.15.72:0/708626178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c107a50 0x7f763c109e40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 --2- 172.21.15.72:0/708626178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c1027f0 0x7f763c104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 --2- 172.21.15.72:0/708626178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 0x7f763c107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 -- 172.21.15.72:0/708626178 >> 172.21.15.72:0/708626178 conn(0x7f763c0fc140 msgr2=0x7f763c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 -- 172.21.15.72:0/708626178 shutdown_connections 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.036+0000 7f7641a00700 1 -- 172.21.15.72:0/708626178 wait complete. 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.037+0000 7f7641a00700 1 Processor -- start 2024-03-11T19:22:42.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 -- start start 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 0x7f763c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c107a50 0x7f763c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1a1b50 con 0x7f763c1027f0 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1a1c90 con 0x7f763c107a50 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f7641a00700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1a1e20 con 0x7f763c105120 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f763aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f763aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59940/0 (socket says 172.21.15.72:59940) 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f763aa00700 1 -- 172.21.15.72:0/2072430074 learned_addr learned my addr 172.21.15.72:0/2072430074 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f763a000700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 0x7f763c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.038+0000 7f763b400700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c107a50 0x7f763c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 -- 172.21.15.72:0/2072430074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 msgr2=0x7f763c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 0x7f763c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 -- 172.21.15.72:0/2072430074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c107a50 msgr2=0x7f763c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c107a50 0x7f763c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 -- 172.21.15.72:0/2072430074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c009520 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f763aa00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f762c00eb60 tx=0x7f762c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00cbd0 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f762c004500 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7634009d00 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c003ea0 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.039+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c1a24c0 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.040+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c1095d0 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.042+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f762c004000 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.044+0000 7f7633400700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 0x7f76200a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.045+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f762c014070 con 0x7f763c1027f0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.045+0000 7f763a000700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 0x7f76200a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:42.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.045+0000 7f763a000700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 0x7f76200a4000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7628009c60 tx=0x7f7628009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:42.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.049+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f762c09a4a0 con 0x7f763c1027f0 2024-03-11T19:22:42.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.271+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f763c068930 con 0x7f763c1027f0 2024-03-11T19:22:42.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.274+0000 7f7633400700 1 -- 172.21.15.72:0/2072430074 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) v1 ==== 56+0+5631 (secure 0 0 0) 0x7f762c0042b0 con 0x7f763c1027f0 2024-03-11T19:22:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 94 2024-03-11T19:22:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:fsid d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:22:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-03-11T19:10:24.349979+0000 2024-03-11T19:22:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-03-11T19:21:32.299563+0000 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:crush_version 27 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:full_ratio 0.95 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:backfillfull_ratio 0.9 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:nearfull_ratio 0.85 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:require_min_compat_client luminous 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:min_compat_client luminous 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:require_osd_release squid 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:stretch_mode_enabled false 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-11T19:22:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:max_osd 12 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6802/1634362166,v1:172.21.15.72:6803/1634362166] [v2:172.21.15.72:6804/1634362166,v1:172.21.15.72:6805/1634362166] exists,up 9cbac0ed-5153-4a8d-b65d-ac1a30323f24 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6800/1421287783,v1:172.21.15.106:6801/1421287783] [v2:172.21.15.106:6802/1421287783,v1:172.21.15.106:6803/1421287783] exists,up 7963ed34-6f58-4b33-bb70-cc1c532602f2 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6800/1986223287,v1:172.21.15.143:6801/1986223287] [v2:172.21.15.143:6802/1986223287,v1:172.21.15.143:6803/1986223287] exists,up 843038a5-98bb-4739-8a50-a0ef1fe16d3d 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 up in weight 1 up_from 26 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6810/3085969755,v1:172.21.15.72:6811/3085969755] [v2:172.21.15.72:6812/3085969755,v1:172.21.15.72:6813/3085969755] exists,up 59081b5d-c259-41da-89c9-0e99ed857bed 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 up in weight 1 up_from 34 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6808/1536676428,v1:172.21.15.106:6809/1536676428] [v2:172.21.15.106:6810/1536676428,v1:172.21.15.106:6811/1536676428] exists,up 55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6808/1302557280,v1:172.21.15.143:6809/1302557280] [v2:172.21.15.143:6810/1302557280,v1:172.21.15.143:6811/1302557280] exists,up 316bc6e8-8a64-4578-9251-779c72149fbe 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6818/2847471062,v1:172.21.15.72:6819/2847471062] [v2:172.21.15.72:6820/2847471062,v1:172.21.15.72:6821/2847471062] exists,up 725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6816/4113260481,v1:172.21.15.106:6817/4113260481] [v2:172.21.15.106:6818/4113260481,v1:172.21.15.106:6819/4113260481] exists,up 8358cb46-e68c-49bd-9add-390afbd8ef8e 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 up in weight 1 up_from 60 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6816/1425068959,v1:172.21.15.143:6817/1425068959] [v2:172.21.15.143:6818/1425068959,v1:172.21.15.143:6819/1425068959] exists,up 4c936701-2ea3-476a-abad-ecaaf2bdc9e4 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6826/98146808,v1:172.21.15.72:6827/98146808] [v2:172.21.15.72:6828/98146808,v1:172.21.15.72:6829/98146808] exists,up 7124f938-a535-4c94-8959-2931abf1fdd0 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6824/3578514696,v1:172.21.15.106:6825/3578514696] [v2:172.21.15.106:6826/3578514696,v1:172.21.15.106:6827/3578514696] exists,up c3cca719-676d-4904-b3be-3f42aef54ae0 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6824/3564968375,v1:172.21.15.143:6825/3564968375] [v2:172.21.15.143:6826/3564968375,v1:172.21.15.143:6827/3564968375] exists,up cbf1e6e6-c003-4c61-a3d0-a1760ed65098 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.4 [2,11] 2024-03-11T19:22:42.282 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.5 [8,11] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.2 [5,6] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.e [4,8] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.34 [4,9] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.38 [3,9] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.3c [4,9] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 4.5 [9,10] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 4.8 [5,8] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 4.e [9,11] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 4.1c [9,8] 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/1618271736 expires 2024-03-12T19:11:29.909764+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/1095172224 expires 2024-03-12T19:10:56.562024+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/2331163134 expires 2024-03-12T19:10:56.562024+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/2769969675 expires 2024-03-12T19:11:29.909764+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/3661300568 expires 2024-03-12T19:10:56.562024+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/2769969675 expires 2024-03-12T19:11:29.909764+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/1095172224 expires 2024-03-12T19:10:56.562024+0000 2024-03-11T19:22:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/618300141 expires 2024-03-12T19:11:29.909764+0000 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/2881891200 expires 2024-03-12T19:11:29.909764+0000 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/3531203560 expires 2024-03-12T19:10:56.562024+0000 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 msgr2=0x7f76200a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 0x7f76200a4000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7628009c60 tx=0x7f7628009260 comp rx=0 tx=0).stop 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 msgr2=0x7f763c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f762c00eb60 tx=0x7f762c00ee70 comp rx=0 tx=0).stop 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 shutdown_connections 2024-03-11T19:22:42.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f76200a1b40 0x7f76200a4000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c107a50 0x7f763c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c1027f0 0x7f763c199fc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 --2- 172.21.15.72:0/2072430074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c105120 0x7f763c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 >> 172.21.15.72:0/2072430074 conn(0x7f763c0fc140 msgr2=0x7f763c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 shutdown_connections 2024-03-11T19:22:42.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:42.277+0000 7f7641a00700 1 -- 172.21.15.72:0/2072430074 wait complete. 2024-03-11T19:22:43.241 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-11T19:22:43.251 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:43 smithi072 ceph-mon[31695]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:43.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:43 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2072430074' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:22:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 -- 172.21.15.72:0/2960935353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce181077c0 msgr2=0x7fce18107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 --2- 172.21.15.72:0/2960935353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce181077c0 0x7fce18107be0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fce08009900 tx=0x7fce0802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 -- 172.21.15.72:0/2960935353 shutdown_connections 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 --2- 172.21.15.72:0/2960935353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce18109370 0x7fce1810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 --2- 172.21.15.72:0/2960935353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 --2- 172.21.15.72:0/2960935353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce181077c0 0x7fce18107be0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 -- 172.21.15.72:0/2960935353 >> 172.21.15.72:0/2960935353 conn(0x7fce18076790 msgr2=0x7fce18078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 -- 172.21.15.72:0/2960935353 shutdown_connections 2024-03-11T19:22:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.464+0000 7fce1ce00640 1 -- 172.21.15.72:0/2960935353 wait complete. 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.465+0000 7fce1ce00640 1 Processor -- start 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.465+0000 7fce1ce00640 1 -- start start 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce181077c0 0x7fce1819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce1819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce181a5370 con 0x7fce18109370 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce181a54e0 con 0x7fce181089b0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce1ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce181a56a0 con 0x7fce181077c0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce16800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce1819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce16800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59972/0 (socket says 172.21.15.72:59972) 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce16800640 1 -- 172.21.15.72:0/3964633249 learned_addr learned my addr 172.21.15.72:0/3964633249 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce15e00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce181077c0 0x7fce1819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 -- 172.21.15.72:0/3964633249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce181077c0 msgr2=0x7fce1819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce181077c0 0x7fce1819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 -- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 msgr2=0x7fce181a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce0fe00640 1 -- 172.21.15.72:0/3964633249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08009560 con 0x7fce181089b0 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.466+0000 7fce16800640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:43.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce0fe00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce1819ddc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e9f0 tx=0x7fce0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000ccc0 con 0x7fce181089b0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce0000ce20 con 0x7fce181089b0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce181a5980 con 0x7fce181089b0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00018920 con 0x7fce181089b0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.467+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce181a5e90 con 0x7fce181089b0 2024-03-11T19:22:43.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.468+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1810ea80 con 0x7fce181089b0 2024-03-11T19:22:43.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.470+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fce00018a80 con 0x7fce181089b0 2024-03-11T19:22:43.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.471+0000 7fce0ea00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 0x7fcdfc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.472+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fce00014070 con 0x7fce181089b0 2024-03-11T19:22:43.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.472+0000 7fce15e00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 0x7fcdfc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.472+0000 7fce15e00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 0x7fcdfc0a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce0802c240 tx=0x7fce0802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:43.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.474+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce00096960 con 0x7fce181089b0 2024-03-11T19:22:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:43 smithi106 ceph-mon[35889]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:43.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:43 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2072430074' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:22:43.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.670+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce18113410 con 0x7fce181089b0 2024-03-11T19:22:43.676 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.672+0000 7fce0ea00640 1 -- 172.21.15.72:0/3964633249 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7fce0009a020 con 0x7fce181089b0 2024-03-11T19:22:43.676 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:43.677 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:43.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 msgr2=0x7fcdfc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 0x7fcdfc0a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce0802c240 tx=0x7fce0802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 msgr2=0x7fce1819ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce1819ddc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e9f0 tx=0x7fce0000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 shutdown_connections 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fcdfc0a1a30 0x7fcdfc0a3ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce18109370 0x7fce181a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce181089b0 0x7fce1819ddc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 --2- 172.21.15.72:0/3964633249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce181077c0 0x7fce1819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 >> 172.21.15.72:0/3964633249 conn(0x7fce18076790 msgr2=0x7fce1810bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.674+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 shutdown_connections 2024-03-11T19:22:43.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.675+0000 7fce1ce00640 1 -- 172.21.15.72:0/3964633249 wait complete. 2024-03-11T19:22:43.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:43 smithi143 ceph-mon[36013]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:43.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:43 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2072430074' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:22:43.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:22:43.700 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:22:43.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 -- 172.21.15.72:0/2568640763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41045b0 msgr2=0x7f5be4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 --2- 172.21.15.72:0/2568640763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41045b0 0x7f5be4104a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009900 tx=0x7f5bcc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:43.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 -- 172.21.15.72:0/2568640763 shutdown_connections 2024-03-11T19:22:43.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 --2- 172.21.15.72:0/2568640763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be4104f70 0x7f5be410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 --2- 172.21.15.72:0/2568640763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41045b0 0x7f5be4104a30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 --2- 172.21.15.72:0/2568640763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be41033c0 0x7f5be41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 -- 172.21.15.72:0/2568640763 >> 172.21.15.72:0/2568640763 conn(0x7f5be40fe930 msgr2=0x7f5be4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:43.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 -- 172.21.15.72:0/2568640763 shutdown_connections 2024-03-11T19:22:43.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.908+0000 7f5bea800640 1 -- 172.21.15.72:0/2568640763 wait complete. 2024-03-11T19:22:43.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.909+0000 7f5bea800640 1 Processor -- start 2024-03-11T19:22:43.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.909+0000 7f5bea800640 1 -- start start 2024-03-11T19:22:43.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.909+0000 7f5bea800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.909+0000 7f5bea800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be41045b0 0x7f5be4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.909+0000 7f5bea800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be4104f70 0x7f5be41a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5bea800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41a0f20 con 0x7f5be41033c0 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5bea800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41a1090 con 0x7f5be4104f70 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5bea800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41a1250 con 0x7f5be41045b0 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59984/0 (socket says 172.21.15.72:59984) 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 -- 172.21.15.72:0/3761181199 learned_addr learned my addr 172.21.15.72:0/3761181199 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5bdaa00640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be41045b0 0x7f5be4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3e00640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be4104f70 0x7f5be41a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 -- 172.21.15.72:0/3761181199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be41045b0 msgr2=0x7f5be4199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be41045b0 0x7f5be4199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 -- 172.21.15.72:0/3761181199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be4104f70 msgr2=0x7f5be41a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:43.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be4104f70 0x7f5be41a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.910+0000 7f5be3400640 1 -- 172.21.15.72:0/3761181199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc009560 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5be3400640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9c0 tx=0x7f5bd400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4009700 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bd400cd80 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be41a1530 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4010430 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.911+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be41a1a10 con 0x7f5be41033c0 2024-03-11T19:22:43.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.912+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be410a6e0 con 0x7f5be41033c0 2024-03-11T19:22:43.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.914+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5bd4003680 con 0x7f5be41033c0 2024-03-11T19:22:43.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.915+0000 7f5be0c00640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 0x7f5bc80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:43.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.915+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f5bd4014070 con 0x7f5be41033c0 2024-03-11T19:22:43.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.916+0000 7f5bdaa00640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 0x7f5bc80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:43.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.916+0000 7f5bdaa00640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 0x7f5bc80a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009230 tx=0x7f5bcc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:43.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:43.919+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bd40969c0 con 0x7f5be41033c0 2024-03-11T19:22:44.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.114+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5be402ce40 con 0x7f5be41033c0 2024-03-11T19:22:44.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.115+0000 7f5be0c00640 1 -- 172.21.15.72:0/3761181199 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bd409a080 con 0x7f5be41033c0 2024-03-11T19:22:44.120 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:22:44.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 msgr2=0x7f5bc80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 0x7f5bc80a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009230 tx=0x7f5bcc005fd0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 msgr2=0x7f5be4199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9c0 tx=0x7f5bd400ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 shutdown_connections 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f5bc80a1a30 0x7f5bc80a3ef0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5be4104f70 0x7f5be41a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5be41045b0 0x7f5be4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 --2- 172.21.15.72:0/3761181199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be41033c0 0x7f5be4199430 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 >> 172.21.15.72:0/3761181199 conn(0x7f5be40fe930 msgr2=0x7f5be41078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 shutdown_connections 2024-03-11T19:22:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.119+0000 7f5bea800640 1 -- 172.21.15.72:0/3761181199 wait complete. 2024-03-11T19:22:44.142 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:22:44.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.350+0000 7f1481600640 1 -- 172.21.15.72:0/2074880806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c108990 msgr2=0x7f147c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 --2- 172.21.15.72:0/2074880806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c108990 0x7f147c108e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1464009860 tx=0x7f146402bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:44.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 -- 172.21.15.72:0/2074880806 shutdown_connections 2024-03-11T19:22:44.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 --2- 172.21.15.72:0/2074880806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c109350 0x7f147c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 --2- 172.21.15.72:0/2074880806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c108990 0x7f147c108e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 --2- 172.21.15.72:0/2074880806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c1077a0 0x7f147c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 -- 172.21.15.72:0/2074880806 >> 172.21.15.72:0/2074880806 conn(0x7f147c076740 msgr2=0x7f147c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:44.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 -- 172.21.15.72:0/2074880806 shutdown_connections 2024-03-11T19:22:44.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.351+0000 7f1481600640 1 -- 172.21.15.72:0/2074880806 wait complete. 2024-03-11T19:22:44.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.352+0000 7f1481600640 1 Processor -- start 2024-03-11T19:22:44.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.352+0000 7f1481600640 1 -- start start 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.352+0000 7f1481600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.352+0000 7f1481600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c108990 0x7f147c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f1481600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c109350 0x7f147c1a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f1481600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c1a5340 con 0x7f147c1077a0 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f1481600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c1a54b0 con 0x7f147c109350 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f1481600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c1a5670 con 0x7f147c108990 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59994/0 (socket says 172.21.15.72:59994) 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 -- 172.21.15.72:0/239907036 learned_addr learned my addr 172.21.15.72:0/239907036 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:44.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f1472000640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c108990 0x7f147c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147b400640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c109350 0x7f147c1a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 -- 172.21.15.72:0/239907036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c108990 msgr2=0x7f147c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c108990 0x7f147c19dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 -- 172.21.15.72:0/239907036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c109350 msgr2=0x7f147c1a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c109350 0x7f147c1a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.353+0000 7f147aa00640 1 -- 172.21.15.72:0/239907036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c009560 con 0x7f147c1077a0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f147aa00640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f146c00e9a0 tx=0x7f146c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00cd80 con 0x7f147c1077a0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f146c010910 con 0x7f147c1077a0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464009490 con 0x7f147c1077a0 2024-03-11T19:22:44.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c018980 con 0x7f147c1077a0 2024-03-11T19:22:44.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.354+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c1a5ca0 con 0x7f147c1077a0 2024-03-11T19:22:44.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.355+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c10eac0 con 0x7f147c1077a0 2024-03-11T19:22:44.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.357+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f146c010a80 con 0x7f147c1077a0 2024-03-11T19:22:44.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.358+0000 7f1473e00640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 0x7f14600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.359+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f146c014070 con 0x7f147c1077a0 2024-03-11T19:22:44.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.359+0000 7f1472000640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 0x7f14600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.360+0000 7f1472000640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 0x7f14600a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f146402c180 tx=0x7f1464034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:44.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.363+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f146c096b60 con 0x7f147c1077a0 2024-03-11T19:22:44.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:44 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3964633249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:44.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:44 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3761181199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:44.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.560+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f147c02cda0 con 0x7f147c1077a0 2024-03-11T19:22:44.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.561+0000 7f1473e00640 1 -- 172.21.15.72:0/239907036 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f146c09a220 con 0x7f147c1077a0 2024-03-11T19:22:44.566 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 msgr2=0x7f14600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 0x7f14600a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f146402c180 tx=0x7f1464034c00 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 msgr2=0x7f147c19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f146c00e9a0 tx=0x7f146c00ee60 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 shutdown_connections 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f14600a1a30 0x7f14600a3ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147c109350 0x7f147c1a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147c108990 0x7f147c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 --2- 172.21.15.72:0/239907036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f147c1077a0 0x7f147c19d850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 >> 172.21.15.72:0/239907036 conn(0x7f147c076740 msgr2=0x7f147c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 shutdown_connections 2024-03-11T19:22:44.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.564+0000 7f1481600640 1 -- 172.21.15.72:0/239907036 wait complete. 2024-03-11T19:22:44.588 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:22:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:44 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3964633249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:44 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3761181199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:44 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3964633249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:44 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3761181199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.802+0000 7f3738200640 1 -- 172.21.15.72:0/3589085428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730109350 msgr2=0x7f373010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.802+0000 7f3738200640 1 --2- 172.21.15.72:0/3589085428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730109350 0x7f373010fbe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3728009ca0 tx=0x7f372802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 -- 172.21.15.72:0/3589085428 shutdown_connections 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 --2- 172.21.15.72:0/3589085428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730109350 0x7f373010fbe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 --2- 172.21.15.72:0/3589085428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730108990 0x7f3730108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 --2- 172.21.15.72:0/3589085428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 0x7f3730107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 -- 172.21.15.72:0/3589085428 >> 172.21.15.72:0/3589085428 conn(0x7f3730076740 msgr2=0x7f3730078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:44.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 -- 172.21.15.72:0/3589085428 shutdown_connections 2024-03-11T19:22:44.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.803+0000 7f3738200640 1 -- 172.21.15.72:0/3589085428 wait complete. 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 Processor -- start 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 -- start start 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 0x7f373019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 0x7f373019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301a52b0 con 0x7f3730109350 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301a5420 con 0x7f3730108990 2024-03-11T19:22:44.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.804+0000 7f3738200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301a55e0 con 0x7f37301077a0 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3736200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 0x7f373019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3735800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 0x7f373019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3736200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60016/0 (socket says 172.21.15.72:60016) 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3736200640 1 -- 172.21.15.72:0/1879818922 learned_addr learned my addr 172.21.15.72:0/1879818922 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 -- 172.21.15.72:0/1879818922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 msgr2=0x7f373019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 0x7f373019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 -- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 msgr2=0x7f37301a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 -- 172.21.15.72:0/1879818922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728009900 con 0x7f3730108990 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3736200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.805+0000 7f3734e00640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 0x7f373019dd00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.806+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00ccc0 con 0x7f3730108990 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.806+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f371c00ce20 con 0x7f3730108990 2024-03-11T19:22:44.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.806+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c018920 con 0x7f3730108990 2024-03-11T19:22:44.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.806+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37301a58c0 con 0x7f3730108990 2024-03-11T19:22:44.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.806+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37301a5d80 con 0x7f3730108990 2024-03-11T19:22:44.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.807+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373010eac0 con 0x7f3730108990 2024-03-11T19:22:44.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.809+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f371c018a80 con 0x7f3730108990 2024-03-11T19:22:44.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.810+0000 7f372e000640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 0x7f37180a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:44.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.811+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f371c014070 con 0x7f3730108990 2024-03-11T19:22:44.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.811+0000 7f3735800640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 0x7f37180a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:44.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.811+0000 7f3735800640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 0x7f37180a3f70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f37301087f0 tx=0x7f372400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:44.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:44.813+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f371c096b40 con 0x7f3730108990 2024-03-11T19:22:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.009+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3730113410 con 0x7f3730108990 2024-03-11T19:22:45.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.010+0000 7f372e000640 1 -- 172.21.15.72:0/1879818922 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f371c09a200 con 0x7f3730108990 2024-03-11T19:22:45.014 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 msgr2=0x7f37180a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 0x7f37180a3f70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f37301087f0 tx=0x7f372400b390 comp rx=0 tx=0).stop 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 msgr2=0x7f373019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 0x7f373019dd00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 shutdown_connections 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f37180a1ab0 0x7f37180a3f70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3730109350 0x7f37301a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3730108990 0x7f373019dd00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 --2- 172.21.15.72:0/1879818922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37301077a0 0x7f373019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.012+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 >> 172.21.15.72:0/1879818922 conn(0x7f3730076740 msgr2=0x7f373010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:45.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.013+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 shutdown_connections 2024-03-11T19:22:45.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.013+0000 7f3738200640 1 -- 172.21.15.72:0/1879818922 wait complete. 2024-03-11T19:22:45.035 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:22:45.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 -- 172.21.15.72:0/3837895758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c104f70 msgr2=0x7f456c10b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 --2- 172.21.15.72:0/3837895758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c104f70 0x7f456c10b800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4564009be0 tx=0x7f456402bef0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 -- 172.21.15.72:0/3837895758 shutdown_connections 2024-03-11T19:22:45.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 --2- 172.21.15.72:0/3837895758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c104f70 0x7f456c10b800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 --2- 172.21.15.72:0/3837895758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c1045b0 0x7f456c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 --2- 172.21.15.72:0/3837895758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1033c0 0x7f456c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 -- 172.21.15.72:0/3837895758 >> 172.21.15.72:0/3837895758 conn(0x7f456c0fe930 msgr2=0x7f456c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:45.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 -- 172.21.15.72:0/3837895758 shutdown_connections 2024-03-11T19:22:45.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.244+0000 7f4573c00640 1 -- 172.21.15.72:0/3837895758 wait complete. 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 Processor -- start 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 -- start start 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1045b0 0x7f456c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c104f70 0x7f456c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c1a0f50 con 0x7f456c1033c0 2024-03-11T19:22:45.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c1a10c0 con 0x7f456c1045b0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.245+0000 7f4573c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c1a1280 con 0x7f456c104f70 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60036/0 (socket says 172.21.15.72:60036) 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 -- 172.21.15.72:0/1567016014 learned_addr learned my addr 172.21.15.72:0/1567016014 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4563e00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1045b0 0x7f456c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c104f70 0x7f456c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 -- 172.21.15.72:0/1567016014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c104f70 msgr2=0x7f456c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c104f70 0x7f456c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 -- 172.21.15.72:0/1567016014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1045b0 msgr2=0x7f456c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1045b0 0x7f456c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 -- 172.21.15.72:0/1567016014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009560 con 0x7f456c1033c0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.246+0000 7f4571200640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e9c0 tx=0x7f455c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.247+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c009700 con 0x7f456c1033c0 2024-03-11T19:22:45.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.247+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f455c00cd80 con 0x7f456c1033c0 2024-03-11T19:22:45.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.247+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4564009840 con 0x7f456c1033c0 2024-03-11T19:22:45.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.247+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c010430 con 0x7f456c1033c0 2024-03-11T19:22:45.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.247+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c1a18e0 con 0x7f456c1033c0 2024-03-11T19:22:45.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.248+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c10a610 con 0x7f456c1033c0 2024-03-11T19:22:45.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.250+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f455c003680 con 0x7f456c1033c0 2024-03-11T19:22:45.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.251+0000 7f456a000640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 0x7f45500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.251+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f455c014070 con 0x7f456c1033c0 2024-03-11T19:22:45.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.252+0000 7f4563e00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 0x7f45500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.252+0000 7f4563e00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 0x7f45500a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f456c19a970 tx=0x7f4554009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:45.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.253+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f455c096c00 con 0x7f456c1033c0 2024-03-11T19:22:45.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:45 smithi072 ceph-mon[31695]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:45.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:45 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/239907036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:45 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1879818922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.460+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f456c10edd0 con 0x7f456c1033c0 2024-03-11T19:22:45.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.461+0000 7f456a000640 1 -- 172.21.15.72:0/1567016014 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f455c09a2c0 con 0x7f456c1033c0 2024-03-11T19:22:45.466 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:45.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 msgr2=0x7f45500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 0x7f45500a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f456c19a970 tx=0x7f4554009260 comp rx=0 tx=0).stop 2024-03-11T19:22:45.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 msgr2=0x7f456c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f455c00e9c0 tx=0x7f455c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 shutdown_connections 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f45500a1a30 0x7f45500a3ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f456c104f70 0x7f456c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f456c1045b0 0x7f456c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 --2- 172.21.15.72:0/1567016014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f456c1033c0 0x7f456c199460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 >> 172.21.15.72:0/1567016014 conn(0x7f456c0fe930 msgr2=0x7f456c1077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 shutdown_connections 2024-03-11T19:22:45.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.464+0000 7f4573c00640 1 -- 172.21.15.72:0/1567016014 wait complete. 2024-03-11T19:22:45.488 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:22:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:45 smithi106 ceph-mon[35889]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:45 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/239907036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:45 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1879818922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:45 smithi143 ceph-mon[36013]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:45 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/239907036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:45 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1879818922' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:45.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 -- 172.21.15.72:0/3239411562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 msgr2=0x7f6174108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 --2- 172.21.15.72:0/3239411562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f6174108e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6164009900 tx=0x7f616402bc50 comp rx=0 tx=0).stop 2024-03-11T19:22:45.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 -- 172.21.15.72:0/3239411562 shutdown_connections 2024-03-11T19:22:45.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 --2- 172.21.15.72:0/3239411562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6174109370 0x7f617410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 --2- 172.21.15.72:0/3239411562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f6174108e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 --2- 172.21.15.72:0/3239411562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61741077c0 0x7f6174107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 -- 172.21.15.72:0/3239411562 >> 172.21.15.72:0/3239411562 conn(0x7f6174076790 msgr2=0x7f6174078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 -- 172.21.15.72:0/3239411562 shutdown_connections 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.694+0000 7f617c600640 1 -- 172.21.15.72:0/3239411562 wait complete. 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.695+0000 7f617c600640 1 Processor -- start 2024-03-11T19:22:45.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.695+0000 7f617c600640 1 -- start start 2024-03-11T19:22:45.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f617419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6174109370 0x7f61741a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a5360 con 0x7f61741089b0 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a54d0 con 0x7f6174109370 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a5690 con 0x7f61741077c0 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f6179c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f6179200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f617419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f6179c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45358/0 (socket says 172.21.15.72:45358) 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f6179c00640 1 -- 172.21.15.72:0/859224452 learned_addr learned my addr 172.21.15.72:0/859224452 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:45.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.696+0000 7f617a600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6174109370 0x7f61741a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 -- 172.21.15.72:0/859224452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6174109370 msgr2=0x7f61741a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6174109370 0x7f61741a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 -- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 msgr2=0x7f617419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.702 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f617419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 -- 172.21.15.72:0/859224452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c009840 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179200640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f617419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6179c00640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9c0 tx=0x7f616800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168009700 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f616800ccc0 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164009560 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.697+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168010430 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.698+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741a5d10 con 0x7f61741077c0 2024-03-11T19:22:45.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.699+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617410eae0 con 0x7f61741077c0 2024-03-11T19:22:45.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.701+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6168003f60 con 0x7f61741077c0 2024-03-11T19:22:45.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.702+0000 7f6172000640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 0x7f615c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:45.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.702+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6168014070 con 0x7f61741077c0 2024-03-11T19:22:45.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.702+0000 7f6179200640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 0x7f615c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:45.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.703+0000 7f6179200640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 0x7f615c0a3f70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6164009230 tx=0x7f6164005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:45.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.704+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6168096980 con 0x7f61741077c0 2024-03-11T19:22:45.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.940+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61741131a0 con 0x7f61741077c0 2024-03-11T19:22:45.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.941+0000 7f6172000640 1 -- 172.21.15.72:0/859224452 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f616809a040 con 0x7f61741077c0 2024-03-11T19:22:45.946 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:45.946 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:22:45.946 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.944+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 msgr2=0x7f615c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.944+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 0x7f615c0a3f70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6164009230 tx=0x7f6164005fd0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.944+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 msgr2=0x7f617419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.944+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9c0 tx=0x7f616800edc0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 shutdown_connections 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f615c0a1ab0 0x7f615c0a3f70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6174109370 0x7f61741a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61741089b0 0x7f617419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 --2- 172.21.15.72:0/859224452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61741077c0 0x7f617419d870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:45.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 >> 172.21.15.72:0/859224452 conn(0x7f6174076790 msgr2=0x7f617410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:45.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 shutdown_connections 2024-03-11T19:22:45.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:45.945+0000 7f617c600640 1 -- 172.21.15.72:0/859224452 wait complete. 2024-03-11T19:22:45.969 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.176+0000 7fd5f3a00640 1 -- 172.21.15.72:0/108451674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1089b0 msgr2=0x7fd5ec108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.176+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/108451674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec108e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009900 tx=0x7fd5d802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 -- 172.21.15.72:0/108451674 shutdown_connections 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/108451674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/108451674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec108e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/108451674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 -- 172.21.15.72:0/108451674 >> 172.21.15.72:0/108451674 conn(0x7fd5ec076790 msgr2=0x7fd5ec078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:46.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 -- 172.21.15.72:0/108451674 shutdown_connections 2024-03-11T19:22:46.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.177+0000 7fd5f3a00640 1 -- 172.21.15.72:0/108451674 wait complete. 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 Processor -- start 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 -- start start 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a5320 con 0x7fd5ec109370 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a5490 con 0x7fd5ec1077c0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a5650 con 0x7fd5ec1089b0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f1a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f1a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60076/0 (socket says 172.21.15.72:60076) 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5f1a00640 1 -- 172.21.15.72:0/2689705071 learned_addr learned my addr 172.21.15.72:0/2689705071 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.178+0000 7fd5ebe00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 -- 172.21.15.72:0/2689705071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 msgr2=0x7fd5ec19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 -- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 msgr2=0x7fd5ec1a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 -- 172.21.15.72:0/2689705071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d8009560 con 0x7fd5ec1077c0 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5ebe00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:46.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5f1000640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec19d830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.179+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000ccc0 con 0x7fd5ec1077c0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.180+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5e000ce20 con 0x7fd5ec1077c0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.180+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec1a5930 con 0x7fd5ec1077c0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.180+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0010430 con 0x7fd5ec1077c0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.180+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec1a5e10 con 0x7fd5ec1077c0 2024-03-11T19:22:46.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.181+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec10ea80 con 0x7fd5ec1077c0 2024-03-11T19:22:46.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.183+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd5e0004000 con 0x7fd5ec1077c0 2024-03-11T19:22:46.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.184+0000 7fd5e9600640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 0x7fd5d40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.184+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fd5e0014070 con 0x7fd5ec1077c0 2024-03-11T19:22:46.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.184+0000 7fd5ebe00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 0x7fd5d40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.185+0000 7fd5ebe00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 0x7fd5d40a3f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000c00 tx=0x7fd5d802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:46.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.186+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5e0096a20 con 0x7fd5ec1077c0 2024-03-11T19:22:46.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.391+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5ec113210 con 0x7fd5ec1077c0 2024-03-11T19:22:46.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.393+0000 7fd5e9600640 1 -- 172.21.15.72:0/2689705071 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7fd5e00aa0a0 con 0x7fd5ec1077c0 2024-03-11T19:22:46.398 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:46.398 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.396+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 msgr2=0x7fd5d40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.396+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 0x7fd5d40a3f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000c00 tx=0x7fd5d802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.396+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 msgr2=0x7fd5ec19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.396+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec19d830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 shutdown_connections 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fd5d40a1ab0 0x7fd5d40a3f70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5ec109370 0x7fd5ec1a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec1089b0 0x7fd5ec19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 --2- 172.21.15.72:0/2689705071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec1077c0 0x7fd5ec19d830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 >> 172.21.15.72:0/2689705071 conn(0x7fd5ec076790 msgr2=0x7fd5ec10bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:46.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 shutdown_connections 2024-03-11T19:22:46.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.397+0000 7fd5f3a00640 1 -- 172.21.15.72:0/2689705071 wait complete. 2024-03-11T19:22:46.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:22:46.421 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:22:46.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.628+0000 7fddb0600640 1 -- 172.21.15.72:0/1612170043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 msgr2=0x7fdda8104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.628+0000 7fddb0600640 1 --2- 172.21.15.72:0/1612170043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8104a10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009860 tx=0x7fdd9c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:46.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 -- 172.21.15.72:0/1612170043 shutdown_connections 2024-03-11T19:22:46.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 --2- 172.21.15.72:0/1612170043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 --2- 172.21.15.72:0/1612170043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8104a10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 --2- 172.21.15.72:0/1612170043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 0x7fdda81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 -- 172.21.15.72:0/1612170043 >> 172.21.15.72:0/1612170043 conn(0x7fdda80fe930 msgr2=0x7fdda8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:46.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 -- 172.21.15.72:0/1612170043 shutdown_connections 2024-03-11T19:22:46.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.629+0000 7fddb0600640 1 -- 172.21.15.72:0/1612170043 wait complete. 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 Processor -- start 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 -- start start 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 0x7fdda8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda81a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda81a0f30 con 0x7fdda8104590 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda81a10a0 con 0x7fdda8104f50 2024-03-11T19:22:46.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.630+0000 7fddb0600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda81a1260 con 0x7fdda81033a0 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60086/0 (socket says 172.21.15.72:60086) 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 -- 172.21.15.72:0/3092625016 learned_addr learned my addr 172.21.15.72:0/3092625016 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddadc00640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 0x7fdda8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddae600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda81a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 -- 172.21.15.72:0/3092625016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 msgr2=0x7fdda8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 0x7fdda8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 -- 172.21.15.72:0/3092625016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 msgr2=0x7fdda81a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda81a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 -- 172.21.15.72:0/3092625016 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c009490 con 0x7fdda8104590 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddae600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda81a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.631+0000 7fddad200640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c034b10 tx=0x7fdd9c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.632+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c005020 con 0x7fdda8104590 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.632+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd9c02c510 con 0x7fdda8104590 2024-03-11T19:22:46.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.632+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c004a60 con 0x7fdda8104590 2024-03-11T19:22:46.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.632+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda81a14e0 con 0x7fdda8104590 2024-03-11T19:22:46.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.632+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda81a1850 con 0x7fdda8104590 2024-03-11T19:22:46.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.633+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda810a6c0 con 0x7fdda8104590 2024-03-11T19:22:46.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.635+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdd9c03b040 con 0x7fdda8104590 2024-03-11T19:22:46.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.636+0000 7fdda6000640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 0x7fdd900a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:46.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.637+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fdd9c0ef100 con 0x7fdda8104590 2024-03-11T19:22:46.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.637+0000 7fddadc00640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 0x7fdd900a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:46.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.637+0000 7fddadc00640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 0x7fdd900a3f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdda81043f0 tx=0x7fdd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:46.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.641+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd9c0b7f30 con 0x7fdda8104590 2024-03-11T19:22:46.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:46 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1567016014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:46.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:46 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/859224452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:46.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1567016014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:46.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:46 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/859224452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:46.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:46 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1567016014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:46.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:46 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/859224452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:46.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.837+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdda802ce40 con 0x7fdda8104590 2024-03-11T19:22:46.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.838+0000 7fdda6000640 1 -- 172.21.15.72:0/3092625016 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd9c0bb5f0 con 0x7fdda8104590 2024-03-11T19:22:46.842 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 msgr2=0x7fdd900a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 0x7fdd900a3f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdda81043f0 tx=0x7fdd98005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 msgr2=0x7fdda8199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c034b10 tx=0x7fdd9c0045c0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 shutdown_connections 2024-03-11T19:22:46.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fdd900a1ab0 0x7fdd900a3f70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda8104f50 0x7fdda81a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdda8104590 0x7fdda8199980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 --2- 172.21.15.72:0/3092625016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdda81033a0 0x7fdda8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 >> 172.21.15.72:0/3092625016 conn(0x7fdda80fe930 msgr2=0x7fdda8107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 shutdown_connections 2024-03-11T19:22:46.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:46.840+0000 7fddb0600640 1 -- 172.21.15.72:0/3092625016 wait complete. 2024-03-11T19:22:46.865 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 -- 172.21.15.72:0/2062263098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 msgr2=0x7f52d410fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 --2- 172.21.15.72:0/2062263098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d410fbe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009ca0 tx=0x7f52cc02c100 comp rx=0 tx=0).stop 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 -- 172.21.15.72:0/2062263098 shutdown_connections 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 --2- 172.21.15.72:0/2062263098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d410fbe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 --2- 172.21.15.72:0/2062263098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 0x7f52d4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 --2- 172.21.15.72:0/2062263098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 -- 172.21.15.72:0/2062263098 >> 172.21.15.72:0/2062263098 conn(0x7f52d4076740 msgr2=0x7f52d4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 -- 172.21.15.72:0/2062263098 shutdown_connections 2024-03-11T19:22:47.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.074+0000 7f52db800640 1 -- 172.21.15.72:0/2062263098 wait complete. 2024-03-11T19:22:47.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.075+0000 7f52db800640 1 Processor -- start 2024-03-11T19:22:47.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.075+0000 7f52db800640 1 -- start start 2024-03-11T19:22:47.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 0x7f52d419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d41a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41a52d0 con 0x7f52d4109350 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41a5440 con 0x7f52d41077a0 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52db800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41a5600 con 0x7f52d4108990 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d9800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d41a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 0x7f52d419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:38562/0 (socket says 172.21.15.72:38562) 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 -- 172.21.15.72:0/127054083 learned_addr learned my addr 172.21.15.72:0/127054083 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 -- 172.21.15.72:0/127054083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 msgr2=0x7f52d419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 0x7f52d419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 -- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 msgr2=0x7f52d41a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d41a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.076+0000 7f52d8e00640 1 -- 172.21.15.72:0/127054083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8009490 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52d8e00640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cde0 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52c8010910 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc009900 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8018a10 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.077+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41a5cb0 con 0x7f52d41077a0 2024-03-11T19:22:47.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.078+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d410ea60 con 0x7f52d41077a0 2024-03-11T19:22:47.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.080+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f52c8010a80 con 0x7f52d41077a0 2024-03-11T19:22:47.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.082+0000 7f52d1600640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 0x7f52bc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.082+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f52c8014070 con 0x7f52d41077a0 2024-03-11T19:22:47.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.082+0000 7f52d3e00640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 0x7f52bc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.082+0000 7f52d3e00640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 0x7f52bc0a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52d419ecf0 tx=0x7f52c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.084+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52c8097240 con 0x7f52d41077a0 2024-03-11T19:22:47.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.280+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52d4113350 con 0x7f52d41077a0 2024-03-11T19:22:47.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.281+0000 7f52d1600640 1 -- 172.21.15.72:0/127054083 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52c809a900 con 0x7f52d41077a0 2024-03-11T19:22:47.285 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 msgr2=0x7f52bc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 0x7f52bc0a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52d419ecf0 tx=0x7f52c0005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 msgr2=0x7f52d419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 shutdown_connections 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f52bc0a1ab0 0x7f52bc0a3f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52d4109350 0x7f52d41a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52d4108990 0x7f52d419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 --2- 172.21.15.72:0/127054083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d41077a0 0x7f52d419d7e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.283+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 >> 172.21.15.72:0/127054083 conn(0x7f52d4076740 msgr2=0x7f52d410bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.284+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 shutdown_connections 2024-03-11T19:22:47.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.284+0000 7f52db800640 1 -- 172.21.15.72:0/127054083 wait complete. 2024-03-11T19:22:47.306 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:22:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 -- 172.21.15.72:0/3987089735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0075140 msgr2=0x7ff2b0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/3987089735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0075140 0x7ff2b0073590 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009ca0 tx=0x7ff2a002c190 comp rx=0 tx=0).stop 2024-03-11T19:22:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 -- 172.21.15.72:0/3987089735 shutdown_connections 2024-03-11T19:22:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/3987089735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0109040 0x7ff2b010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/3987089735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0073b60 0x7ff2b0073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/3987089735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0075140 0x7ff2b0073590 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 -- 172.21.15.72:0/3987089735 >> 172.21.15.72:0/3987089735 conn(0x7ff2b00fc720 msgr2=0x7ff2b00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 -- 172.21.15.72:0/3987089735 shutdown_connections 2024-03-11T19:22:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.510+0000 7ff2b7e00640 1 -- 172.21.15.72:0/3987089735 wait complete. 2024-03-11T19:22:47.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.511+0000 7ff2b7e00640 1 Processor -- start 2024-03-11T19:22:47.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.511+0000 7ff2b7e00640 1 -- start start 2024-03-11T19:22:47.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 0x7ff2b0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0109040 0x7ff2b01a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01a0f80 con 0x7ff2b0075140 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01a10f0 con 0x7ff2b0073b60 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01a12b0 con 0x7ff2b0109040 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60116/0 (socket says 172.21.15.72:60116) 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 -- 172.21.15.72:0/1886137927 learned_addr learned my addr 172.21.15.72:0/1886137927 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:47.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b5400640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 0x7ff2b0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b5e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0109040 0x7ff2b01a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 -- 172.21.15.72:0/1886137927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0109040 msgr2=0x7ff2b01a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0109040 0x7ff2b01a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 -- 172.21.15.72:0/1886137927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 msgr2=0x7ff2b0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 0x7ff2b0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b4a00640 1 -- 172.21.15.72:0/1886137927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009900 con 0x7ff2b0075140 2024-03-11T19:22:47.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.512+0000 7ff2b5400640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 0x7ff2b0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:47.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2b4a00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c009720 con 0x7ff2b0075140 2024-03-11T19:22:47.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff29c00cd80 con 0x7ff2b0075140 2024-03-11T19:22:47.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b01a1590 con 0x7ff2b0075140 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c010690 con 0x7ff2b0075140 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.513+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b01a1a50 con 0x7ff2b0075140 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.514+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0107e00 con 0x7ff2b0075140 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.517+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff29c0046c0 con 0x7ff2b0075140 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.518+0000 7ff2ae000640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 0x7ff2940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.518+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7ff29c014070 con 0x7ff2b0075140 2024-03-11T19:22:47.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.519+0000 7ff2b5400640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 0x7ff2940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.519+0000 7ff2b5400640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 0x7ff2940a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c7a0 tx=0x7ff2a0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.523+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff29c096980 con 0x7ff2b0075140 2024-03-11T19:22:47.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:47 smithi106 ceph-mon[35889]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:47.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:47 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2689705071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:47.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:47 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3092625016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:47.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:47 smithi072 ceph-mon[31695]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:47.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:47 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2689705071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:47.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:47 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3092625016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:47.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:47 smithi143 ceph-mon[36013]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:47.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:47 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2689705071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:47.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:47 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3092625016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:22:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.719+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2b002cdd0 con 0x7ff2b0075140 2024-03-11T19:22:47.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.719+0000 7ff2ae000640 1 -- 172.21.15.72:0/1886137927 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff29c09a040 con 0x7ff2b0075140 2024-03-11T19:22:47.724 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:22:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 msgr2=0x7ff2940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 0x7ff2940a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c7a0 tx=0x7ff2a0005ef0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 msgr2=0x7ff2b01999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00edf0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 shutdown_connections 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff2940a1a30 0x7ff2940a3ef0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2b0109040 0x7ff2b01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2b0075140 0x7ff2b01999d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 --2- 172.21.15.72:0/1886137927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2b0073b60 0x7ff2b0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 >> 172.21.15.72:0/1886137927 conn(0x7ff2b00fc720 msgr2=0x7ff2b0104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 shutdown_connections 2024-03-11T19:22:47.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.722+0000 7ff2b7e00640 1 -- 172.21.15.72:0/1886137927 wait complete. 2024-03-11T19:22:47.746 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:22:47.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.956+0000 7fb575600640 1 -- 172.21.15.72:0/875711598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 msgr2=0x7fb57010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.956+0000 7fb575600640 1 --2- 172.21.15.72:0/875711598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb57010b7e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb568009900 tx=0x7fb56802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:47.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 -- 172.21.15.72:0/875711598 shutdown_connections 2024-03-11T19:22:47.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 --2- 172.21.15.72:0/875711598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb57010b7e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 --2- 172.21.15.72:0/875711598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb570104590 0x7fb570104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 --2- 172.21.15.72:0/875711598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5701033a0 0x7fb5701037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 -- 172.21.15.72:0/875711598 >> 172.21.15.72:0/875711598 conn(0x7fb5700fe930 msgr2=0x7fb570100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 -- 172.21.15.72:0/875711598 shutdown_connections 2024-03-11T19:22:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.957+0000 7fb575600640 1 -- 172.21.15.72:0/875711598 wait complete. 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 Processor -- start 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 -- start start 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5701033a0 0x7fb5701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb570104590 0x7fb570199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a0eb0 con 0x7fb5701033a0 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a1020 con 0x7fb570104590 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb575600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701a11e0 con 0x7fb570104f50 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb56f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb56f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45440/0 (socket says 172.21.15.72:45440) 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb56f400640 1 -- 172.21.15.72:0/3533698889 learned_addr learned my addr 172.21.15.72:0/3533698889 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb56e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb570104590 0x7fb570199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.958+0000 7fb56ea00640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5701033a0 0x7fb5701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 -- 172.21.15.72:0/3533698889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb570104590 msgr2=0x7fb570199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb570104590 0x7fb570199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 -- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5701033a0 msgr2=0x7fb5701993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5701033a0 0x7fb5701993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 -- 172.21.15.72:0/3533698889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb568009560 con 0x7fb570104f50 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb56f400640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb568009230 tx=0x7fb568004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.959+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568004440 con 0x7fb570104f50 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.960+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5701a1460 con 0x7fb570104f50 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.960+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5680052d0 con 0x7fb570104f50 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.960+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568003cc0 con 0x7fb570104f50 2024-03-11T19:22:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.960+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5701a19c0 con 0x7fb570104f50 2024-03-11T19:22:47.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.961+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57010a6c0 con 0x7fb570104f50 2024-03-11T19:22:47.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.963+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb568005440 con 0x7fb570104f50 2024-03-11T19:22:47.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.964+0000 7fb567400640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 0x7fb5540a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:47.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.964+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fb5680f0160 con 0x7fb570104f50 2024-03-11T19:22:47.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.964+0000 7fb56ea00640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 0x7fb5540a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:47.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.965+0000 7fb56ea00640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 0x7fb5540a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5701043f0 tx=0x7fb560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:47.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:47.966+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5680b8f90 con 0x7fb570104f50 2024-03-11T19:22:48.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.165+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb57010efd0 con 0x7fb570104f50 2024-03-11T19:22:48.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.166+0000 7fb567400640 1 -- 172.21.15.72:0/3533698889 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb5680430a0 con 0x7fb570104f50 2024-03-11T19:22:48.170 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 msgr2=0x7fb5540a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 0x7fb5540a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5701043f0 tx=0x7fb560005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 msgr2=0x7fb5701a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb568009230 tx=0x7fb568004040 comp rx=0 tx=0).stop 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 shutdown_connections 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fb5540a1a30 0x7fb5540a3ef0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb570104f50 0x7fb5701a0970 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb570104590 0x7fb570199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 --2- 172.21.15.72:0/3533698889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5701033a0 0x7fb5701993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 >> 172.21.15.72:0/3533698889 conn(0x7fb5700fe930 msgr2=0x7fb570107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:48.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 shutdown_connections 2024-03-11T19:22:48.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.169+0000 7fb575600640 1 -- 172.21.15.72:0/3533698889 wait complete. 2024-03-11T19:22:48.193 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:22:48.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 -- 172.21.15.72:0/3375679195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 msgr2=0x7fa98c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 --2- 172.21.15.72:0/3375679195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c1095e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa984009ca0 tx=0x7fa98402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 -- 172.21.15.72:0/3375679195 shutdown_connections 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 --2- 172.21.15.72:0/3375679195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c1095e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 --2- 172.21.15.72:0/3375679195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 --2- 172.21.15.72:0/3375679195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 0x7fa98c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 -- 172.21.15.72:0/3375679195 >> 172.21.15.72:0/3375679195 conn(0x7fa98c0fc730 msgr2=0x7fa98c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 -- 172.21.15.72:0/3375679195 shutdown_connections 2024-03-11T19:22:48.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.407+0000 7fa994000640 1 -- 172.21.15.72:0/3375679195 wait complete. 2024-03-11T19:22:48.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.408+0000 7fa994000640 1 Processor -- start 2024-03-11T19:22:48.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.408+0000 7fa994000640 1 -- start start 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 0x7fa98c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c19ec80 con 0x7fa98c102d50 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c19edf0 con 0x7fa98c1011a0 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa994000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c19efb0 con 0x7fa98c102390 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45454/0 (socket says 172.21.15.72:45454) 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 -- 172.21.15.72:0/1450325082 learned_addr learned my addr 172.21.15.72:0/1450325082 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa992000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa991600640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 0x7fa98c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 -- 172.21.15.72:0/1450325082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 msgr2=0x7fa98c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 0x7fa98c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 -- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 msgr2=0x7fa98c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.409+0000 7fa990c00640 1 -- 172.21.15.72:0/1450325082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009900 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa990c00640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9f0 tx=0x7fa97c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c009700 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa97c00cd80 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c018920 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c19f290 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.410+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c19f750 con 0x7fa98c102390 2024-03-11T19:22:48.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.411+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c1084e0 con 0x7fa98c102390 2024-03-11T19:22:48.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.413+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa97c018a80 con 0x7fa98c102390 2024-03-11T19:22:48.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.414+0000 7fa98a000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 0x7fa9740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.415+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fa97c014070 con 0x7fa98c102390 2024-03-11T19:22:48.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.415+0000 7fa991600640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 0x7fa9740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.415+0000 7fa991600640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 0x7fa9740a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa98c1021f0 tx=0x7fa980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:48.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.418+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa97c052d20 con 0x7fa98c102390 2024-03-11T19:22:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/127054083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1886137927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:48 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3533698889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.656+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa98c02cdd0 con 0x7fa98c102390 2024-03-11T19:22:48.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.658+0000 7fa98a000640 1 -- 172.21.15.72:0/1450325082 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fa97c096a30 con 0x7fa98c102390 2024-03-11T19:22:48.662 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:48.662 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:20:15.233431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:22:48.662 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 24 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.660+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 msgr2=0x7fa9740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.660+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 0x7fa9740a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa98c1021f0 tx=0x7fa980005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.660+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 msgr2=0x7fa98c1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.660+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9f0 tx=0x7fa97c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 shutdown_connections 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7fa9740a1a30 0x7fa9740a3ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa98c102d50 0x7fa98c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa98c102390 0x7fa98c1976d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 --2- 172.21.15.72:0/1450325082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa98c1011a0 0x7fa98c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 >> 172.21.15.72:0/1450325082 conn(0x7fa98c0fc730 msgr2=0x7fa98c1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:48.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 shutdown_connections 2024-03-11T19:22:48.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.661+0000 7fa994000640 1 -- 172.21.15.72:0/1450325082 wait complete. 2024-03-11T19:22:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/127054083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1886137927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:48 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3533698889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.685 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:22:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/127054083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1886137927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:48 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3533698889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.888+0000 7f18fe000640 1 -- 172.21.15.72:0/1463964040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81077c0 msgr2=0x7f18f8107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.888+0000 7f18fe000640 1 --2- 172.21.15.72:0/1463964040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81077c0 0x7f18f8107be0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009900 tx=0x7f18e802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 -- 172.21.15.72:0/1463964040 shutdown_connections 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 --2- 172.21.15.72:0/1463964040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 0x7f18f810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 --2- 172.21.15.72:0/1463964040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81089b0 0x7f18f8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 --2- 172.21.15.72:0/1463964040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81077c0 0x7f18f8107be0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 -- 172.21.15.72:0/1463964040 >> 172.21.15.72:0/1463964040 conn(0x7f18f8076790 msgr2=0x7f18f8078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 -- 172.21.15.72:0/1463964040 shutdown_connections 2024-03-11T19:22:48.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.889+0000 7f18fe000640 1 -- 172.21.15.72:0/1463964040 wait complete. 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.890+0000 7f18fe000640 1 Processor -- start 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.890+0000 7f18fe000640 1 -- start start 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.890+0000 7f18fe000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.890+0000 7f18fe000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81089b0 0x7f18f819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18fe000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 0x7f18f81a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18fe000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81a5360 con 0x7f18f81077c0 2024-03-11T19:22:48.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18fe000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81a54d0 con 0x7f18f8109370 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18fe000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81a5690 con 0x7f18f81089b0 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60170/0 (socket says 172.21.15.72:60170) 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 -- 172.21.15.72:0/3371741084 learned_addr learned my addr 172.21.15.72:0/3371741084 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7e00640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 0x7f18f81a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18efe00640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81089b0 0x7f18f819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 -- 172.21.15.72:0/3371741084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81089b0 msgr2=0x7f18f819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81089b0 0x7f18f819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 -- 172.21.15.72:0/3371741084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 msgr2=0x7f18f81a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 0x7f18f81a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:48.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.891+0000 7f18f7400640 1 -- 172.21.15.72:0/3371741084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009560 con 0x7f18f81077c0 2024-03-11T19:22:48.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18f7400640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005ab0 tx=0x7f18e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:48.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004aa0 con 0x7f18f81077c0 2024-03-11T19:22:48.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f81a5910 con 0x7f18f81077c0 2024-03-11T19:22:48.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18e80052d0 con 0x7f18f81077c0 2024-03-11T19:22:48.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8003e80 con 0x7f18f81077c0 2024-03-11T19:22:48.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.892+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f81a5e20 con 0x7f18f81077c0 2024-03-11T19:22:48.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.893+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f810eae0 con 0x7f18f81077c0 2024-03-11T19:22:48.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.895+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f18e8005440 con 0x7f18f81077c0 2024-03-11T19:22:48.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.896+0000 7f18f4c00640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:48.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.897+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f18e80efc30 con 0x7f18f81077c0 2024-03-11T19:22:48.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.897+0000 7f18efe00640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:48.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.897+0000 7f18efe00640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18f819ed80 tx=0x7f18e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:48.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:48.900+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18e80b8a60 con 0x7f18f81077c0 2024-03-11T19:22:49.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.098+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18f802ce40 con 0x7f18f81077c0 2024-03-11T19:22:49.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.100+0000 7f18f4c00640 1 -- 172.21.15.72:0/3371741084 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24713 (secure 0 0 0) 0x7f18e80bc120 con 0x7f18f81077c0 2024-03-11T19:22:49.105 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:22:49.105 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":94,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:21:32.299563+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:22:49.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 msgr2=0x7f18dc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18f819ed80 tx=0x7f18e000b410 comp rx=0 tx=0).stop 2024-03-11T19:22:49.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 msgr2=0x7f18f819d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005ab0 tx=0x7f18e8004270 comp rx=0 tx=0).stop 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 shutdown_connections 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f18dc0a1a30 0x7f18dc0a3ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18f8109370 0x7f18f81a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f81089b0 0x7f18f819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 --2- 172.21.15.72:0/3371741084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18f81077c0 0x7f18f819d870 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.103+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 >> 172.21.15.72:0/3371741084 conn(0x7f18f8076790 msgr2=0x7f18f810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.104+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 shutdown_connections 2024-03-11T19:22:49.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.104+0000 7f18fe000640 1 -- 172.21.15.72:0/3371741084 wait complete. 2024-03-11T19:22:49.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:19:41.837094+0000', 'modified': '2024-03-11T19:20:15.233431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24542, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/2743792791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2743792791}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2743792791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6837/902566146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 902566146}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 902566146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/2982797341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2982797341}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2982797341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.143:6839/3455090526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 3455090526}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 3455090526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.72:6839/3478523060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 3478523060}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 3478523060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/806155003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 806155003}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 806155003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:22:49.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:22:49.129 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 -- 172.21.15.72:0/3485514372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1045b0 msgr2=0x7f33fc104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 --2- 172.21.15.72:0/3485514372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1045b0 0x7f33fc104a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e402bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 -- 172.21.15.72:0/3485514372 shutdown_connections 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 --2- 172.21.15.72:0/3485514372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc104f70 0x7f33fc10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 --2- 172.21.15.72:0/3485514372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1045b0 0x7f33fc104a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 --2- 172.21.15.72:0/3485514372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 -- 172.21.15.72:0/3485514372 >> 172.21.15.72:0/3485514372 conn(0x7f33fc0fe930 msgr2=0x7f33fc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 -- 172.21.15.72:0/3485514372 shutdown_connections 2024-03-11T19:22:49.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.341+0000 7f3403200640 1 -- 172.21.15.72:0/3485514372 wait complete. 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.342+0000 7f3403200640 1 Processor -- start 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 -- start start 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 0x7f33fc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104f70 0x7f33fc1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1a0f30 con 0x7f33fc1045b0 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1a10a0 con 0x7f33fc104f70 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3403200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1a1260 con 0x7f33fc1033c0 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f33fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 0x7f33fc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45474/0 (socket says 172.21.15.72:45474) 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f33fbe00640 1 -- 172.21.15.72:0/1013849761 learned_addr learned my addr 172.21.15.72:0/1013849761 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.343+0000 7f3401200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104f70 0x7f33fc1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 -- 172.21.15.72:0/1013849761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104f70 msgr2=0x7f33fc1a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104f70 0x7f33fc1a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 -- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 msgr2=0x7f33fc199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 0x7f33fc199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 -- 172.21.15.72:0/1013849761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4009560 con 0x7f33fc1033c0 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fb400640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 0x7f33fc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33f400eb70 tx=0x7f33f400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33f8c00640 1 -- 172.21.15.72:0/1013849761 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4003b70 con 0x7f33fc1033c0 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33f8c00640 1 -- 172.21.15.72:0/1013849761 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f400ee70 con 0x7f33fc1033c0 2024-03-11T19:22:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc1a1540 con 0x7f33fc1033c0 2024-03-11T19:22:49.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.344+0000 7f33f8c00640 1 -- 172.21.15.72:0/1013849761 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40107f0 con 0x7f33fc1033c0 2024-03-11T19:22:49.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.345+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc1a19d0 con 0x7f33fc1033c0 2024-03-11T19:22:49.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f33f8c00640 1 -- 172.21.15.72:0/1013849761 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f33f4004020 con 0x7f33fc1033c0 2024-03-11T19:22:49.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 Processor -- start 2024-03-11T19:22:49.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 -- start start 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d182a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d1830e0 0x55852d185570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d0ecb30 con 0x55852d17d580 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d0ea060 con 0x55852d1830e0 2024-03-11T19:22:49.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3405f96740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d0e5840 con 0x55852d17df70 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3401200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d182a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3401200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60206/0 (socket says 172.21.15.72:60206) 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.348+0000 7f3401200640 1 -- 172.21.15.72:0/2016195884 learned_addr learned my addr 172.21.15.72:0/2016195884 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fb400640 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d1830e0 0x55852d185570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 -- 172.21.15.72:0/2016195884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d1830e0 msgr2=0x55852d185570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d1830e0 0x55852d185570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 -- 172.21.15.72:0/2016195884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 msgr2=0x55852d17d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 -- 172.21.15.72:0/2016195884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852d0ad4c0 con 0x55852d17df70 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f3401200640 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d182a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cb4d0 tx=0x7f33f40d1c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480de406efb4cf88 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.349+0000 7f33f8c00640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 0x7f33e00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f33f1600640 1 -- 172.21.15.72:0/2016195884 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40d1e90 con 0x55852d17df70 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3401200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 0x7f33e00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f33f1600640 1 -- 172.21.15.72:0/2016195884 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f4003cd0 con 0x55852d17df70 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f33f1600640 1 -- 172.21.15.72:0/2016195884 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4015c10 con 0x55852d17df70 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f33f8c00640 1 -- 172.21.15.72:0/1013849761 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f33f4014070 con 0x7f33fc1033c0 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3405f96740 1 -- 172.21.15.72:0/2016195884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 msgr2=0x55852d182a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3405f96740 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d182a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cb4d0 tx=0x7f33f40d1c40 comp rx=0 tx=0).stop 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3401200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 0x7f33e00a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55852d17e7d0 tx=0x7f33ec00b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3405f96740 1 -- 172.21.15.72:0/2016195884 shutdown_connections 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.350+0000 7f3405f96740 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d1830e0 0x55852d185570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d182a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 --2- 172.21.15.72:0/2016195884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d17d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 -- 172.21.15.72:0/2016195884 >> 172.21.15.72:0/2016195884 conn(0x55852d178af0 msgr2=0x55852d178ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 -- 172.21.15.72:0/2016195884 shutdown_connections 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 -- 172.21.15.72:0/2016195884 wait complete. 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 -- start start 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.351+0000 7f3405f96740 1 Processor -- start 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d187880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d08aea0 con 0x55852d1830e0 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d08ab40 con 0x55852d17d580 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852ce2df20 con 0x55852d17df70 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3405f96740 10 client.0 _ll_register_callbacks cb 0x55852d09e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3401200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:38682/0 (socket says 172.21.15.72:38682) 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3401200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60208/0 (socket says 172.21.15.72:60208) 2024-03-11T19:22:49.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 -- 172.21.15.72:0/2681954933 learned_addr learned my addr 172.21.15.72:0/2681954933 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fb400640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d187880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 -- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 msgr2=0x55852d187880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d187880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 -- 172.21.15.72:0/2681954933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 msgr2=0x55852d18e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f33fbe00640 1 -- 172.21.15.72:0/2681954933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f40d2040 con 0x55852d17d580 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.352+0000 7f3401200640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33fbe00640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33f409a670 tx=0x7f33f40cde50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc3d596f0ebc8ac server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40d1160 con 0x55852d17d580 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f33f4003e60 con 0x55852d17d580 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55852d109c40 con 0x55852d17d580 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f400b2c0 con 0x55852d17d580 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f3405f96740 10 client.25313 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55852d0e3ec0 con 0x55852d17d580 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55852d109c40 con 0x55852d17d580 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.353+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33f401f060 con 0x55852d17d580 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 10 client.25313 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 10 client.25313 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 20 client.25313 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1934d0 0x55852d197950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 4 client.25313 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 --> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55852cde56c0 con 0x55852d1934d0 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.354+0000 7f33fb400640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1934d0 0x55852d197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.355+0000 7f33fb400640 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1934d0 0x55852d197950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6024c1379980eb68 server_cookie=b1f7f185d246267a in_seq=0 out_seq=0 2024-03-11T19:22:49.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.355+0000 7f33ea000640 10 client.25313 ms_handle_connect on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:49.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.355+0000 7f33ea000640 1 -- 172.21.15.72:0/2681954933 <== mds.2 v2:172.21.15.143:6838/3455090526 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55852cde56c0 con 0x55852d1934d0 2024-03-11T19:22:49.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f33ea000640 10 client.25313 handle_command_reply: tid=0 2024-03-11T19:22:49.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 client.25313 shutdown 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 msgr2=0x55852d187340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33f409a670 tx=0x7f33f40cde50 comp rx=0 tx=0).stop 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 shutdown_connections 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1934d0 0x55852d197950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d1830e0 0x55852d18e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d187880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 --2- 172.21.15.72:0/2681954933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d187340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f33ea000640 0 client.25313 ms_handle_reset on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 >> 172.21.15.72:0/2681954933 conn(0x55852d178af0 msgr2=0x55852d177e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 shutdown_connections 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 1 -- 172.21.15.72:0/2681954933 wait complete. 2024-03-11T19:22:49.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.356+0000 7f3405f96740 20 client.25313 trim_cache size 0 max 16384 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 Processor -- start 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 -- start start 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d18e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d193d60 con 0x55852d178af0 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d1943c0 con 0x55852d17d580 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.373+0000 7f3405f96740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d18f180 con 0x55852d17df70 2024-03-11T19:22:49.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f3401200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f3401200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45508/0 (socket says 172.21.15.72:45508) 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f3401200640 1 -- 172.21.15.72:0/1516060178 learned_addr learned my addr 172.21.15.72:0/1516060178 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60218/0 (socket says 172.21.15.72:60218) 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fb400640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d18e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 -- 172.21.15.72:0/1516060178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 msgr2=0x55852d19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 -- 172.21.15.72:0/1516060178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 msgr2=0x55852d18e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d18e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fbe00640 1 -- 172.21.15.72:0/1516060178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852d0808b0 con 0x55852d178af0 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f33fb400640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d18e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.374+0000 7f3401200640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f33fbe00640 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33f4097340 tx=0x7f33f40cdec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a0b98add6b31e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f33f0c00640 1 -- 172.21.15.72:0/1516060178 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40d6640 con 0x55852d178af0 2024-03-11T19:22:49.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f33f0c00640 1 -- 172.21.15.72:0/1516060178 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f4085040 con 0x55852d178af0 2024-03-11T19:22:49.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f33f0c00640 1 -- 172.21.15.72:0/1516060178 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4003e60 con 0x55852d178af0 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f3405f96740 1 -- 172.21.15.72:0/1516060178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 msgr2=0x55852d19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.375+0000 7f3405f96740 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f33f4097340 tx=0x7f33f40cdec0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 -- 172.21.15.72:0/1516060178 shutdown_connections 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d17df70 0x55852d19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17d580 0x55852d18e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 --2- 172.21.15.72:0/1516060178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d178af0 0x55852d19b970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 -- 172.21.15.72:0/1516060178 >> 172.21.15.72:0/1516060178 conn(0x55852d179240 msgr2=0x55852d182660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 -- 172.21.15.72:0/1516060178 shutdown_connections 2024-03-11T19:22:49.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 -- 172.21.15.72:0/1516060178 wait complete. 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 -- start start 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.376+0000 7f3405f96740 1 Processor -- start 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d178af0 0x55852d18a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17df70 0x55852d18df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d18e450 con 0x55852d17d580 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d18e5c0 con 0x55852d17df70 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852d18e780 con 0x55852d178af0 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3405f96740 10 client.0 _ll_register_callbacks cb 0x7f33ec004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60224/0 (socket says 172.21.15.72:60224) 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 -- 172.21.15.72:0/3669397170 learned_addr learned my addr 172.21.15.72:0/3669397170 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f3401200640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17df70 0x55852d18df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fbe00640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d178af0 0x55852d18a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 -- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d178af0 msgr2=0x55852d18a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d178af0 0x55852d18a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 -- 172.21.15.72:0/3669397170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17df70 msgr2=0x55852d18df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17df70 0x55852d18df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 -- 172.21.15.72:0/3669397170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec004460 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33fb400640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddab899bdb040ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.377+0000 7f33ea000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4045040 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f33e4004250 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55852d194500 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33e4004410 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55852d0e3ec0 con 0x55852d17d580 2024-03-11T19:22:49.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55852d194500 con 0x55852d17d580 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33e403c040 con 0x55852d17d580 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 20 client.15222 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.378+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1830e0 0x55852d1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f3405f96740 4 client.15222 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 --> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55852d193ea0 con 0x55852d1830e0 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f33fbe00640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1830e0 0x55852d1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f33fbe00640 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1830e0 0x55852d1f2f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6689708a785318f5 server_cookie=6193cbedd615277c in_seq=0 out_seq=0 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f33ea000640 10 client.15222 ms_handle_connect on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f33ea000640 1 -- 172.21.15.72:0/3669397170 <== mds.2 v2:172.21.15.143:6838/3455090526 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55852d193ea0 con 0x55852d1830e0 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f33ea000640 10 client.15222 handle_command_reply: tid=0 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.379+0000 7f3405f96740 1 client.15222 shutdown 2024-03-11T19:22:49.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 msgr2=0x55852d18aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e4005b50 comp rx=0 tx=0).stop 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 shutdown_connections 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:6838/3455090526,v1:172.21.15.143:6839/3455090526] conn(0x55852d1830e0 0x55852d1f2f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55852d17df70 0x55852d18df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55852d17d580 0x55852d18aef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f33ea000640 0 client.15222 ms_handle_reset on v2:172.21.15.143:6838/3455090526 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 --2- 172.21.15.72:0/3669397170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55852d178af0 0x55852d18a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 >> 172.21.15.72:0/3669397170 conn(0x55852d179240 msgr2=0x55852d19b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 shutdown_connections 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 1 -- 172.21.15.72:0/3669397170 wait complete. 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.380+0000 7f3405f96740 20 client.15222 trim_cache size 0 max 16384 2024-03-11T19:22:49.389 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24532, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 155.154761076, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 204.558859966 2024-03-11T19:22:49.390 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 msgr2=0x7f33e00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 0x7f33e00a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55852d17e7d0 tx=0x7f33ec00b2e0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 msgr2=0x7f33fc199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33f400eb70 tx=0x7f33f400c6a0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 shutdown_connections 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f33e00a1a30 0x7f33e00a3ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104f70 0x7f33fc1a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33fc1045b0 0x7f33fc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 --2- 172.21.15.72:0/1013849761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33fc1033c0 0x7f33fc199440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 >> 172.21.15.72:0/1013849761 conn(0x7f33fc0fe930 msgr2=0x7f33fc107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 shutdown_connections 2024-03-11T19:22:49.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.381+0000 7f3403200640 1 -- 172.21.15.72:0/1013849761 wait complete. 2024-03-11T19:22:49.400 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:49 smithi072 ceph-mon[31695]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:49.400 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1450325082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.400 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:49 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3371741084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.402 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-11T19:22:49.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 -- 172.21.15.72:0/1182490622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 msgr2=0x7f6f70104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 --2- 172.21.15.72:0/1182490622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70104a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009860 tx=0x7f6f5c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 -- 172.21.15.72:0/1182490622 shutdown_connections 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 --2- 172.21.15.72:0/1182490622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f70104f70 0x7f6f7010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 --2- 172.21.15.72:0/1182490622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70104a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 --2- 172.21.15.72:0/1182490622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f701033c0 0x7f6f701037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 -- 172.21.15.72:0/1182490622 >> 172.21.15.72:0/1182490622 conn(0x7f6f700fe930 msgr2=0x7f6f70100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 -- 172.21.15.72:0/1182490622 shutdown_connections 2024-03-11T19:22:49.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.604+0000 7f6f78400640 1 -- 172.21.15.72:0/1182490622 wait complete. 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 Processor -- start 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 -- start start 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f701033c0 0x7f6f70199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 0x7f6f701a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f701a0f20 con 0x7f6f70104f70 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f701a1090 con 0x7f6f701033c0 2024-03-11T19:22:49.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.605+0000 7f6f78400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f701a1250 con 0x7f6f701045b0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f76400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 0x7f6f701a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45536/0 (socket says 172.21.15.72:45536) 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 -- 172.21.15.72:0/1922606289 learned_addr learned my addr 172.21.15.72:0/1922606289 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75a00640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f701033c0 0x7f6f70199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 -- 172.21.15.72:0/1922606289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f701033c0 msgr2=0x7f6f70199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f701033c0 0x7f6f70199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 -- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 msgr2=0x7f6f701a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 0x7f6f701a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 -- 172.21.15.72:0/1922606289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009560 con 0x7f6f701045b0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f76400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 0x7f6f701a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.606+0000 7f6f75000640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005940 tx=0x7f6f5c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.607+0000 7f6f6e000640 1 -- 172.21.15.72:0/1922606289 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c02c510 con 0x7f6f701045b0 2024-03-11T19:22:49.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.607+0000 7f6f6e000640 1 -- 172.21.15.72:0/1922606289 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c02cb50 con 0x7f6f701045b0 2024-03-11T19:22:49.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.607+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c009490 con 0x7f6f701045b0 2024-03-11T19:22:49.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.607+0000 7f6f6e000640 1 -- 172.21.15.72:0/1922606289 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c032860 con 0x7f6f701045b0 2024-03-11T19:22:49.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.607+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f701a16b0 con 0x7f6f701045b0 2024-03-11T19:22:49.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 Processor -- start 2024-03-11T19:22:49.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 -- start start 2024-03-11T19:22:49.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f6e000640 1 -- 172.21.15.72:0/1922606289 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6f5c03b040 con 0x7f6f701045b0 2024-03-11T19:22:49.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f9728aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f9728d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f971f4b30 con 0x560f97285f70 2024-03-11T19:22:49.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f971f2060 con 0x560f9728b0e0 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.611+0000 7f6f7b084740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f971ed840 con 0x560f97285580 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f9728aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f75000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f9728d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45550/0 (socket says 172.21.15.72:45550) 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 -- 172.21.15.72:0/1473815435 learned_addr learned my addr 172.21.15.72:0/1473815435 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 -- 172.21.15.72:0/1473815435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 msgr2=0x560f9728d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f9728d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 -- 172.21.15.72:0/1473815435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 msgr2=0x560f9728aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f9728aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 -- 172.21.15.72:0/1473815435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f971b54c0 con 0x560f97285580 2024-03-11T19:22:49.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.612+0000 7f6f76400640 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f680117e0 tx=0x7f6f68011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a00d506b141ee6a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f62a00640 1 -- 172.21.15.72:0/1473815435 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68011310 con 0x560f97285580 2024-03-11T19:22:49.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f62a00640 1 -- 172.21.15.72:0/1473815435 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f68003e50 con 0x560f97285580 2024-03-11T19:22:49.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f62a00640 1 -- 172.21.15.72:0/1473815435 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004560 con 0x560f97285580 2024-03-11T19:22:49.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f6e000640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 0x7f6f580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f6e000640 1 -- 172.21.15.72:0/1922606289 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6f5c0ef420 con 0x7f6f701045b0 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.613+0000 7f6f75a00640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 0x7f6f580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- 172.21.15.72:0/1473815435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 msgr2=0x560f972859a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f680117e0 tx=0x7f6f68011ca0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- 172.21.15.72:0/1473815435 shutdown_connections 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f9728d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f9728aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 --2- 172.21.15.72:0/1473815435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f972859a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- 172.21.15.72:0/1473815435 >> 172.21.15.72:0/1473815435 conn(0x560f97280af0 msgr2=0x560f97280ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- 172.21.15.72:0/1473815435 shutdown_connections 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- 172.21.15.72:0/1473815435 wait complete. 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f75a00640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 0x7f6f580a3ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x560f97281a60 tx=0x7f6f64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 -- start start 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.614+0000 7f6f7b084740 1 Processor -- start 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f97285f70 0x560f9728f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f9728b0e0 0x560f972969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97192ea0 con 0x560f97285580 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97192b40 con 0x560f97285f70 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f96f35f20 con 0x560f9728b0e0 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f7b084740 10 client.0 _ll_register_callbacks cb 0x560f971a6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60252/0 (socket says 172.21.15.72:60252) 2024-03-11T19:22:49.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f75a00640 1 -- 172.21.15.72:0/660414587 learned_addr learned my addr 172.21.15.72:0/660414587 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f76400640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f9728b0e0 0x560f972969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.615+0000 7f6f75000640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f97285f70 0x560f9728f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 -- 172.21.15.72:0/660414587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f9728b0e0 msgr2=0x560f972969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f9728b0e0 0x560f972969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 -- 172.21.15.72:0/660414587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f97285f70 msgr2=0x560f9728f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f97285f70 0x560f9728f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 -- 172.21.15.72:0/660414587 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680114c0 con 0x560f97285580 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f75a00640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401d9c0 tx=0x7f6f64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a56e0dd7260e7f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f57400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64022070 con 0x560f97285580 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6f6400f480 con 0x560f97285580 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.616+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f97211c40 con 0x560f97285580 2024-03-11T19:22:49.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f64017070 con 0x560f97285580 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 10 client.15234 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f971ebec0 con 0x560f97285580 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560f97211c40 con 0x560f97285580 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f6401c760 con 0x560f97285580 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 10 client.15234 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 20 client.15234 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f9729baa0 0x560f9729ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 4 client.15234 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:49.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.617+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 --> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f96eed6c0 con 0x560f9729baa0 2024-03-11T19:22:49.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.618+0000 7f6f75000640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f9729baa0 0x560f9729ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.618+0000 7f6f75000640 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f9729baa0 0x560f9729ad50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a30ca112181579a4 server_cookie=ec000cc6be910917 in_seq=0 out_seq=0 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.618+0000 7f6f57400640 10 client.15234 ms_handle_connect on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f57400640 1 -- 172.21.15.72:0/660414587 <== mds.1 v2:172.21.15.72:6838/3478523060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f96eed6c0 con 0x560f9729baa0 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f57400640 10 client.15234 handle_command_reply: tid=0 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 client.15234 shutdown 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 msgr2=0x560f9728f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401d9c0 tx=0x7f6f64004730 comp rx=0 tx=0).stop 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 shutdown_connections 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f9729baa0 0x560f9729ad50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f9728b0e0 0x560f972969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f97285f70 0x560f9728f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f57400640 0 client.15234 ms_handle_reset on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 --2- 172.21.15.72:0/660414587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f9728f430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 >> 172.21.15.72:0/660414587 conn(0x560f97280af0 msgr2=0x560f9727fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 shutdown_connections 2024-03-11T19:22:49.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 1 -- 172.21.15.72:0/660414587 wait complete. 2024-03-11T19:22:49.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.619+0000 7f6f7b084740 20 client.15234 trim_cache size 0 max 16384 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 Processor -- start 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 -- start start 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285f70 0x560f97296940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97286800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9729da80 con 0x560f97285580 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f96eed6c0 con 0x560f9728b0e0 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.636+0000 7f6f7b084740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97211c40 con 0x560f97285f70 2024-03-11T19:22:49.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60268/0 (socket says 172.21.15.72:60268) 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 -- 172.21.15.72:0/79090716 learned_addr learned my addr 172.21.15.72:0/79090716 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75000640 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285f70 0x560f97296940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f76400640 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97286800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 -- 172.21.15.72:0/79090716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285f70 msgr2=0x560f97296940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285f70 0x560f97296940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 -- 172.21.15.72:0/79090716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 msgr2=0x560f97286800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97286800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.637+0000 7f6f75a00640 1 -- 172.21.15.72:0/79090716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f971888b0 con 0x560f97285580 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.638+0000 7f6f75a00640 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f640058c0 tx=0x7f6f64012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55639ca6a82cfc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.638+0000 7f6f62000640 1 -- 172.21.15.72:0/79090716 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64022070 con 0x560f97285580 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.638+0000 7f6f62000640 1 -- 172.21.15.72:0/79090716 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f6401c880 con 0x560f97285580 2024-03-11T19:22:49.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.638+0000 7f6f62000640 1 -- 172.21.15.72:0/79090716 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400f940 con 0x560f97285580 2024-03-11T19:22:49.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- 172.21.15.72:0/79090716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 msgr2=0x560f97284460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f640058c0 tx=0x7f6f64012d50 comp rx=0 tx=0).stop 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- 172.21.15.72:0/79090716 shutdown_connections 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97286800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285f70 0x560f97296940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 --2- 172.21.15.72:0/79090716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285580 0x560f97284460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- 172.21.15.72:0/79090716 >> 172.21.15.72:0/79090716 conn(0x560f97280af0 msgr2=0x560f97283d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- 172.21.15.72:0/79090716 shutdown_connections 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- 172.21.15.72:0/79090716 wait complete. 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 -- start start 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.639+0000 7f6f7b084740 1 Processor -- start 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f97293a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97294f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97295460 con 0x560f97285f70 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f972955d0 con 0x560f9728b0e0 2024-03-11T19:22:49.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97295790 con 0x560f97285580 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f7b084740 10 client.0 _ll_register_callbacks cb 0x7f6f6401d880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60274/0 (socket says 172.21.15.72:60274) 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f97293a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 -- 172.21.15.72:0/4167402639 learned_addr learned my addr 172.21.15.72:0/4167402639 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f76400640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97294f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 -- 172.21.15.72:0/4167402639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 msgr2=0x560f97293a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f97293a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 -- 172.21.15.72:0/4167402639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 msgr2=0x560f97294f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97294f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75000640 1 -- 172.21.15.72:0/4167402639 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f971ebec0 con 0x560f97285f70 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f75a00640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f97293a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.640+0000 7f6f76400640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97294f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f75000640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0bc050 tx=0x7f6f5c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b905346525583ce server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f57400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c003e40 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6f5c0bc0c0 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f68003e50 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f5c0c0160 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f7b084740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f971ae510 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.641+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f68003e50 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f5c003780 con 0x560f97285f70 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 10 client.15240 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 20 client.15240 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f972f6ef0 0x560f972f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 4 client.15240 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 --> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560f96f3e270 con 0x560f972f6ef0 2024-03-11T19:22:49.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f75a00640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f972f6ef0 0x560f972f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f75a00640 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f972f6ef0 0x560f972f36b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17ac01c8dca2b909 server_cookie=2cca99479f009669 in_seq=0 out_seq=0 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.642+0000 7f6f57400640 10 client.15240 ms_handle_connect on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f57400640 1 -- 172.21.15.72:0/4167402639 <== mds.1 v2:172.21.15.72:6838/3478523060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560f96f3e270 con 0x560f972f6ef0 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f57400640 10 client.15240 handle_command_reply: tid=0 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 client.15240 shutdown 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 msgr2=0x560f97293fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0bc050 tx=0x7f6f5c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 shutdown_connections 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:6838/3478523060,v1:172.21.15.72:6839/3478523060] conn(0x560f972f6ef0 0x560f972f36b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560f9728b0e0 0x560f97294f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560f97285f70 0x560f97293fb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f57400640 0 client.15240 ms_handle_reset on v2:172.21.15.72:6838/3478523060 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 --2- 172.21.15.72:0/4167402639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560f97285580 0x560f97293a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 >> 172.21.15.72:0/4167402639 conn(0x560f97280af0 msgr2=0x560f972845e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 shutdown_connections 2024-03-11T19:22:49.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 1 -- 172.21.15.72:0/4167402639 wait complete. 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.643+0000 7f6f7b084740 20 client.15240 trim_cache size 0 max 16384 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24542, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 156.42672254499999, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:49.653 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 200.587016718 2024-03-11T19:22:49.654 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:49.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 msgr2=0x7f6f580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 0x7f6f580a3ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x560f97281a60 tx=0x7f6f64009260 comp rx=0 tx=0).stop 2024-03-11T19:22:49.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 msgr2=0x7f6f70199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005940 tx=0x7f6f5c0045e0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 shutdown_connections 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f6f580a1a30 0x7f6f580a3ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f70104f70 0x7f6f701a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f701045b0 0x7f6f70199970 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 --2- 172.21.15.72:0/1922606289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f701033c0 0x7f6f70199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 >> 172.21.15.72:0/1922606289 conn(0x7f6f700fe930 msgr2=0x7f6f70107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 shutdown_connections 2024-03-11T19:22:49.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.644+0000 7f6f78400640 1 -- 172.21.15.72:0/1922606289 wait complete. 2024-03-11T19:22:49.655 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:49 smithi106 ceph-mon[35889]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:49.655 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1450325082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.655 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:49 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3371741084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.665 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:22:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:49 smithi143 ceph-mon[36013]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:22:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1450325082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:49 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3371741084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 -- 172.21.15.72:0/2858339921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 msgr2=0x7ff044108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 --2- 172.21.15.72:0/2858339921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff044108e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009860 tx=0x7ff02c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 -- 172.21.15.72:0/2858339921 shutdown_connections 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 --2- 172.21.15.72:0/2858339921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 0x7ff04410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 --2- 172.21.15.72:0/2858339921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff044108e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 --2- 172.21.15.72:0/2858339921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff044107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 -- 172.21.15.72:0/2858339921 >> 172.21.15.72:0/2858339921 conn(0x7ff044076740 msgr2=0x7ff044078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 -- 172.21.15.72:0/2858339921 shutdown_connections 2024-03-11T19:22:49.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.857+0000 7ff04a400640 1 -- 172.21.15.72:0/2858339921 wait complete. 2024-03-11T19:22:49.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 Processor -- start 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 -- start start 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff04419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 0x7ff0441a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441a5340 con 0x7ff0441077a0 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441a54b0 con 0x7ff044109350 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff04a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441a5670 con 0x7ff044108990 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff043400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff043400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60298/0 (socket says 172.21.15.72:60298) 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff043400640 1 -- 172.21.15.72:0/3470443080 learned_addr learned my addr 172.21.15.72:0/3470443080 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.859+0000 7ff03aa00640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff04419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043e00640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 0x7ff0441a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 -- 172.21.15.72:0/3470443080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 msgr2=0x7ff04419dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff04419dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 -- 172.21.15.72:0/3470443080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 msgr2=0x7ff0441a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 0x7ff0441a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 -- 172.21.15.72:0/3470443080 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c009490 con 0x7ff0441077a0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff043400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff03400e9c0 tx=0x7ff03400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff040c00640 1 -- 172.21.15.72:0/3470443080 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034009700 con 0x7ff0441077a0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0441a5950 con 0x7ff0441077a0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.860+0000 7ff040c00640 1 -- 172.21.15.72:0/3470443080 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff03400cd80 con 0x7ff0441077a0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.861+0000 7ff040c00640 1 -- 172.21.15.72:0/3470443080 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034018430 con 0x7ff0441077a0 2024-03-11T19:22:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.861+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0441a5e30 con 0x7ff0441077a0 2024-03-11T19:22:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 Processor -- start 2024-03-11T19:22:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff040c00640 1 -- 172.21.15.72:0/3470443080 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff034003680 con 0x7ff0441077a0 2024-03-11T19:22:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 -- start start 2024-03-11T19:22:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 0x558d224d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224d8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2243fb30 con 0x558d224d0580 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2243d060 con 0x558d224d60e0 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.864+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d22438840 con 0x558d224d0f70 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 0x558d224d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60304/0 (socket says 172.21.15.72:60304) 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 0x558d224d5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:45592/0 (socket says 172.21.15.72:45592) 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 -- 172.21.15.72:0/828650460 learned_addr learned my addr 172.21.15.72:0/828650460 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff03aa00640 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 -- 172.21.15.72:0/828650460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 msgr2=0x558d224d5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 0x558d224d5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 -- 172.21.15.72:0/828650460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 msgr2=0x558d224d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 -- 172.21.15.72:0/828650460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d224004c0 con 0x558d224d0580 2024-03-11T19:22:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff040c00640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 0x7ff0280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.865+0000 7ff043e00640 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e580 tx=0x7ff03c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fe02a804ca684c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.866+0000 7ff040c00640 1 -- 172.21.15.72:0/3470443080 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7ff034014070 con 0x7ff0441077a0 2024-03-11T19:22:49.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.866+0000 7ff043400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 0x7ff0280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.866+0000 7ff038c00640 1 -- 172.21.15.72:0/828650460 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01b070 con 0x558d224d0580 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.866+0000 7ff038c00640 1 -- 172.21.15.72:0/828650460 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff03c003e50 con 0x558d224d0580 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.866+0000 7ff038c00640 1 -- 172.21.15.72:0/828650460 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c013040 con 0x558d224d0580 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- 172.21.15.72:0/828650460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 msgr2=0x558d224d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff043400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 0x7ff0280a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0340cb5b0 tx=0x7ff0340d3550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e580 tx=0x7ff03c0095c0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- 172.21.15.72:0/828650460 shutdown_connections 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0f70 0x558d224d5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/828650460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0580 0x558d224d09a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- 172.21.15.72:0/828650460 >> 172.21.15.72:0/828650460 conn(0x558d224cbaf0 msgr2=0x558d224cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- 172.21.15.72:0/828650460 shutdown_connections 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- 172.21.15.72:0/828650460 wait complete. 2024-03-11T19:22:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 -- start start 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.867+0000 7ff04cfe5740 1 Processor -- start 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0580 0x558d224da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 0x558d224e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d223ddea0 con 0x558d224d0f70 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d223ddb40 con 0x558d224d0580 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d22180f20 con 0x558d224d60e0 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff04cfe5740 10 client.0 _ll_register_callbacks cb 0x558d223f1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60308/0 (socket says 172.21.15.72:60308) 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff03aa00640 1 -- 172.21.15.72:0/128327558 learned_addr learned my addr 172.21.15.72:0/128327558 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff043e00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0580 0x558d224da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.868+0000 7ff043e00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 0x558d224e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 -- 172.21.15.72:0/128327558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 msgr2=0x558d224e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 0x558d224e18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 -- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0580 msgr2=0x558d224da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0580 0x558d224da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 -- 172.21.15.72:0/128327558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c009d60 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff043e00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 0x558d224e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff03aa00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02c180 tx=0x7ff02c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a931f9617ce095 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff031600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c02c510 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7ff02c02c670 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.869+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d2245cc40 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c02c830 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 10 client.15252 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558d22436ec0 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d2245cc40 con 0x558d224d0f70 2024-03-11T19:22:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff02c032620 con 0x558d224d0f70 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 10 client.15252 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 20 client.15252 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d224e6300 0x558d224e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 4 client.15252 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.870+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d221386c0 con 0x558d224e6300 2024-03-11T19:22:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.871+0000 7ff043e00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d224e6300 0x558d224e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.871+0000 7ff043e00640 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d224e6300 0x558d224e5c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff308769996c06b server_cookie=c167f110986a7725 in_seq=0 out_seq=0 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.871+0000 7ff031600640 10 client.15252 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff031600640 1 -- 172.21.15.72:0/128327558 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d221386c0 con 0x558d224e6300 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff031600640 10 client.15252 handle_command_reply: tid=0 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 client.15252 shutdown 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 msgr2=0x558d224da8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02c180 tx=0x7ff02c005db0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 shutdown_connections 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d224e6300 0x558d224e5c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d60e0 0x558d224e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224da8b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff031600640 0 client.15252 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/128327558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0580 0x558d224da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.872+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 >> 172.21.15.72:0/128327558 conn(0x558d224cbaf0 msgr2=0x558d224cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.873+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 shutdown_connections 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.873+0000 7ff04cfe5740 1 -- 172.21.15.72:0/128327558 wait complete. 2024-03-11T19:22:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.873+0000 7ff04cfe5740 20 client.15252 trim_cache size 0 max 16384 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 Processor -- start 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 -- start start 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224e6b70 con 0x558d224d0f70 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224db590 con 0x558d224d60e0 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224d9bc0 con 0x558d224d0580 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff043e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224e18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60316/0 (socket says 172.21.15.72:60316) 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff03aa00640 1 -- 172.21.15.72:0/543396244 learned_addr learned my addr 172.21.15.72:0/543396244 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.890+0000 7ff043e00640 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 -- 172.21.15.72:0/543396244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 msgr2=0x558d224da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 -- 172.21.15.72:0/543396244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 msgr2=0x558d224e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 -- 172.21.15.72:0/543396244 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d223d38b0 con 0x558d224d0580 2024-03-11T19:22:49.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff043e00640 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224cbed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00dfd0 tx=0x7ff03c009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeedf5bf558dcf13 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff033e00640 1 -- 172.21.15.72:0/543396244 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c013040 con 0x558d224d0580 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff033e00640 1 -- 172.21.15.72:0/543396244 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff03c02a070 con 0x558d224d0580 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.891+0000 7ff033e00640 1 -- 172.21.15.72:0/543396244 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01b040 con 0x558d224d0580 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- 172.21.15.72:0/543396244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 msgr2=0x558d224cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224cbed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00dfd0 tx=0x7ff03c009cc0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- 172.21.15.72:0/543396244 shutdown_connections 2024-03-11T19:22:49.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d60e0 0x558d224da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d0f70 0x558d224e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/543396244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224cbed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- 172.21.15.72:0/543396244 >> 172.21.15.72:0/543396244 conn(0x558d224cbaf0 msgr2=0x558d224c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- 172.21.15.72:0/543396244 shutdown_connections 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- 172.21.15.72:0/543396244 wait complete. 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 -- start start 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.892+0000 7ff04cfe5740 1 Processor -- start 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d60e0 0x558d224e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224e21b0 con 0x558d224d60e0 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224e06a0 con 0x558d224d0f70 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d224e0860 con 0x558d224d0580 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff04cfe5740 10 client.0 _ll_register_callbacks cb 0x7ff03c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff043e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff043e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d60e0 0x558d224e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:38810/0 (socket says 172.21.15.72:38810) 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 -- 172.21.15.72:0/1059909805 learned_addr learned my addr 172.21.15.72:0/1059909805 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 -- 172.21.15.72:0/1059909805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 msgr2=0x558d224dec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224dec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 -- 172.21.15.72:0/1059909805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d60e0 msgr2=0x558d224e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d60e0 0x558d224e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff03aa00640 1 -- 172.21.15.72:0/1059909805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d224004c0 con 0x558d224d0f70 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.893+0000 7ff043e00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff03aa00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff02c036010 tx=0x7ff02c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4d054675fa0f6b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff031600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c032bf0 con 0x558d224d0f70 2024-03-11T19:22:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7ff02c032600 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d224e6cb0 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c0327c0 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff04cfe5740 10 client.25355 fetch_fsmap learned FSMap version 24 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.894+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558d223f9510 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d224e6cb0 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff02c02c710 con 0x558d224d0f70 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 10 client.25355 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 10 client.25355 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 20 client.25355 populate_metadata read hostname 'smithi072' 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d22542050 0x558d2253e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 4 client.25355 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:22:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7ff03c003e50 con 0x558d22542050 2024-03-11T19:22:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff043e00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d22542050 0x558d2253e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff043e00640 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d22542050 0x558d2253e550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6860cc17b4a108f server_cookie=7f97c9cd0e264907 in_seq=0 out_seq=0 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.895+0000 7ff031600640 10 client.25355 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff031600640 1 -- 172.21.15.72:0/1059909805 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7ff03c003e50 con 0x558d22542050 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff031600640 10 client.25355 handle_command_reply: tid=0 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 client.25355 shutdown 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 msgr2=0x558d224df1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff02c036010 tx=0x7ff02c0058d0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 shutdown_connections 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558d22542050 0x558d2253e550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d224d60e0 0x558d224e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558d224d0f70 0x558d224df1a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff031600640 0 client.25355 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 --2- 172.21.15.72:0/1059909805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d224d0580 0x558d224dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 >> 172.21.15.72:0/1059909805 conn(0x558d224cbaf0 msgr2=0x558d224cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 shutdown_connections 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 1 -- 172.21.15.72:0/1059909805 wait complete. 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.896+0000 7ff04cfe5740 20 client.25355 trim_cache size 0 max 16384 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "d6472314-dfda-11ee-95c8-87774f69a715", 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24572, 2024-03-11T19:22:49.905 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 187.144524566, 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 24, 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 94, 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 193.635247569 2024-03-11T19:22:49.906 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 msgr2=0x7ff0280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 0x7ff0280a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0340cb5b0 tx=0x7ff0340d3550 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 msgr2=0x7ff04419d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff03400e9c0 tx=0x7ff03400ee80 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 shutdown_connections 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7ff0280a1a30 0x7ff0280a3ef0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff044109350 0x7ff0441a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff044108990 0x7ff04419dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 --2- 172.21.15.72:0/3470443080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0441077a0 0x7ff04419d850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 >> 172.21.15.72:0/3470443080 conn(0x7ff044076740 msgr2=0x7ff04410bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 shutdown_connections 2024-03-11T19:22:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:49.897+0000 7ff04a400640 1 -- 172.21.15.72:0/3470443080 wait complete. 2024-03-11T19:22:49.918 INFO:teuthology.run_tasks:Running task kclient... 2024-03-11T19:22:49.932 INFO:tasks.kclient:Mounting kernel clients... 2024-03-11T19:22:49.932 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-11T19:22:49.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:49.932 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-03-11T19:22:49.980 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-11T19:22:49.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:49.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:22:50.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:50.077 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-03-11T19:22:50.212 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-11T19:22:50.213 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T19:22:50.213 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T19:22:50.213 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-11T19:22:50.641 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:22:51.344 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:51 smithi072 ceph-mon[31695]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:51.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:51 smithi106 ceph-mon[35889]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:51.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:51 smithi143 ceph-mon[36013]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:22:53.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 -- 172.21.15.72:0/1664896175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe810e720 msgr2=0x7f0fe8110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:53.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 --2- 172.21.15.72:0/1664896175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe810e720 0x7f0fe8110b10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009f90 tx=0x7f0fe000b330 comp rx=0 tx=0).stop 2024-03-11T19:22:53.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 -- 172.21.15.72:0/1664896175 shutdown_connections 2024-03-11T19:22:53.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 --2- 172.21.15.72:0/1664896175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe810e720 0x7f0fe8110b10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 --2- 172.21.15.72:0/1664896175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe8069920 0x7f0fe8069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 --2- 172.21.15.72:0/1664896175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe806a240 0x7f0fe810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 -- 172.21.15.72:0/1664896175 >> 172.21.15.72:0/1664896175 conn(0x7f0fe8077aa0 msgr2=0x7f0fe8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:53.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 -- 172.21.15.72:0/1664896175 shutdown_connections 2024-03-11T19:22:53.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.232+0000 7f0fed200700 1 -- 172.21.15.72:0/1664896175 wait complete. 2024-03-11T19:22:53.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.233+0000 7f0fed200700 1 Processor -- start 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 -- start start 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe806a240 0x7f0fe819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 0x7f0fe819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe819f4b0 con 0x7f0fe806a240 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe819f620 con 0x7f0fe810e720 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fed200700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe819f7e0 con 0x7f0fe8069920 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fe5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fe5e00700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:49258/0 (socket says 172.21.15.72:49258) 2024-03-11T19:22:53.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fe5e00700 1 -- 172.21.15.72:0/2403881976 learned_addr learned my addr 172.21.15.72:0/2403881976 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fe5400700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe806a240 0x7f0fe819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.234+0000 7f0fe6800700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 0x7f0fe819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 -- 172.21.15.72:0/2403881976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 msgr2=0x7f0fe819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 0x7f0fe819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 -- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe806a240 msgr2=0x7f0fe819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe806a240 0x7f0fe819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 -- 172.21.15.72:0/2403881976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009c40 con 0x7f0fe8069920 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe6800700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 0x7f0fe819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fe5e00700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9a0 tx=0x7f0fd800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8009790 con 0x7f0fe8069920 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fd8010430 con 0x7f0fe8069920 2024-03-11T19:22:53.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe819fac0 con 0x7f0fe8069920 2024-03-11T19:22:53.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.235+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800cea0 con 0x7f0fe8069920 2024-03-11T19:22:53.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.236+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe81a7320 con 0x7f0fe8069920 2024-03-11T19:22:53.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.237+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe8109940 con 0x7f0fe8069920 2024-03-11T19:22:53.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.238+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0fd8010910 con 0x7f0fe8069920 2024-03-11T19:22:53.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.240+0000 7f0fdea00700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 0x7f0fcc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:53.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.240+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f0fd8014070 con 0x7f0fe8069920 2024-03-11T19:22:53.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.240+0000 7f0fe5400700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 0x7f0fcc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:53.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.241+0000 7f0fe5400700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 0x7f0fcc0a3f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8102160 tx=0x7f0fd4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:53.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.243+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fd8052d20 con 0x7f0fe8069920 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:53 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.515+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0fe819b930 con 0x7f0fe8069920 2024-03-11T19:22:53.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.515+0000 7f0fdea00700 1 -- 172.21.15.72:0/2403881976 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0fd8096840 con 0x7f0fe8069920 2024-03-11T19:22:53.520 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 msgr2=0x7f0fcc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 0x7f0fcc0a3f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8102160 tx=0x7f0fd4009310 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 msgr2=0x7f0fe819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9a0 tx=0x7f0fd800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 shutdown_connections 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f0fcc0a1ac0 0x7f0fcc0a3f80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe810e720 0x7f0fe819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fe8069920 0x7f0fe819a620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 --2- 172.21.15.72:0/2403881976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fe806a240 0x7f0fe819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.518+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 >> 172.21.15.72:0/2403881976 conn(0x7f0fe8077aa0 msgr2=0x7f0fe810fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:53.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.519+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 shutdown_connections 2024-03-11T19:22:53.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:53.519+0000 7f0fed200700 1 -- 172.21.15.72:0/2403881976 wait complete. 2024-03-11T19:22:53.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:53.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:53 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:53.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:53 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:54.558 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:54 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2403881976' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:22:54.607 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-11T19:22:54.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T19:22:54.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2024-03-11T19:22:54.607 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-11T19:22:54.607 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T19:22:54.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-11T19:22:54.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T19:22:54.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T19:22:54.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-11T19:22:54.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:54.608 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2024-03-11T19:22:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:54 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2403881976' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:22:54.651 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 41794sec preferred_lft 41794sec 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2024-03-11T19:22:54.652 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:22:54.653 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T19:22:54.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:54.653 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:22:54.653 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:22:54.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2024-03-11T19:22:54.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2024-03-11T19:22:54.654 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2024-03-11T19:22:54.654 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T19:22:54.654 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:22:54.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:54 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2403881976' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:22:54.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:54.882 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T19:22:54.955 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-03-11T19:22:54.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:54.959 DEBUG:teuthology.orchestra.run.smithi072:> route 2024-03-11T19:22:55.006 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2024-03-11T19:22:55.007 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T19:22:55.007 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T19:22:55.007 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T19:22:55.007 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T19:22:55.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.007 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:22:55.007 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:22:55.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T19:22:55.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T19:22:55.008 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T19:22:55.008 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:22:55.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.173 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-03-11T19:22:55.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.216 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2024-03-11T19:22:55.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.299 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:22:55.299 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:22:55.299 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T19:22:55.299 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-11T19:22:55.299 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:22:55.417 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:55 smithi072 ceph-mon[31695]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:55.417 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.418 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.418 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.418 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:55 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.474 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-11T19:22:55.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.474 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-11T19:22:55.475 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:22:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:55 smithi106 ceph-mon[35889]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:55 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:55 smithi143 ceph-mon[36013]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:22:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:55 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:55.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:55.735 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2024-03-11T19:22:55.735 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-03-11T19:22:55.735 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2024-03-11T19:22:55.735 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T19:22:55.735 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-03-11T19:22:55.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:22:55.874 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-11T19:22:55.914 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-11T19:22:55.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:22:55.914 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-11T19:22:55.999 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-11T19:22:56.000 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-11T19:22:56.000 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/13cd1ee8-743e-48fd-a290-a198de32115e 2024-03-11T19:22:56.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:56.000 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/13cd1ee8-743e-48fd-a290-a198de32115e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:22:56.161 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:22:56.161 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-11T19:22:56.161 INFO:teuthology.orchestra.run.smithi072.stdout:invalid new device string format 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:invalid new device string format 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.72:3300,172.21.15.106:3300,172.21.15.143:3300" 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.72:3300,172.21.15.106:3300,172.21.15.143:3300:/volumes/qa/sv_1/13cd1ee8-743e-48fd-a290-a198de32115e 2024-03-11T19:22:57.040 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=d6472314-dfda-11ee-95c8-87774f69a715" will pass to kernel 2024-03-11T19:22:57.047 INFO:teuthology.orchestra.run.smithi072.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-11T19:22:57.047 INFO:teuthology.orchestra.run.smithi072.stdout: You just mounted a file system that supports labels which does not 2024-03-11T19:22:57.047 INFO:teuthology.orchestra.run.smithi072.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-11T19:22:57.047 INFO:teuthology.orchestra.run.smithi072.stdout: applications will generate AVC messages and not be allowed access to 2024-03-11T19:22:57.047 INFO:teuthology.orchestra.run.smithi072.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-11T19:22:57.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-11T19:22:57.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:57.053 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-11T19:22:57.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:57.142 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:22:57.197 INFO:teuthology.orchestra.run.smithi072.stdout:client15267 2024-03-11T19:22:57.269 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:22:57.269 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:22:57.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:57.320 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:22:57.402 INFO:teuthology.orchestra.run.smithi072.stdout:client15267 2024-03-11T19:22:57.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:57.402 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:22:57.490 INFO:teuthology.orchestra.run.smithi072.stdout:client15267 2024-03-11T19:22:57.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:22:57.490 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6472314-dfda-11ee-95c8-87774f69a715.client15267/status') 2024-03-11T19:22:57.601 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:57 smithi072 ceph-mon[31695]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:57.601 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.601 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:57 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.608 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.15267 (3)192.168.0.1:0/3380840960 2024-03-11T19:22:57.608 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2024-03-11T19:22:57.608 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2024-03-11T19:22:57.609 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2024-03-11T19:22:57.609 INFO:teuthology.orchestra.run.smithi072.stderr:70 bytes copied, 5.5315e-05 s, 1.3 MB/s 2024-03-11T19:22:57.612 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:22:57.623 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:22:57.623 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:22:57.623 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-11T19:22:57.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:57 smithi106 ceph-mon[35889]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:57.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:57 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:57 smithi143 ceph-mon[36013]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:22:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:57 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 -- 172.21.15.72:0/3720534418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81033e0 msgr2=0x7f89e8103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 --2- 172.21.15.72:0/3720534418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81033e0 0x7f89e8103800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009900 tx=0x7f89d802bc30 comp rx=0 tx=0).stop 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 -- 172.21.15.72:0/3720534418 shutdown_connections 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 --2- 172.21.15.72:0/3720534418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e8104f90 0x7f89e810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 --2- 172.21.15.72:0/3720534418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e81045d0 0x7f89e8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 --2- 172.21.15.72:0/3720534418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81033e0 0x7f89e8103800 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.808+0000 7f89ece00640 1 -- 172.21.15.72:0/3720534418 >> 172.21.15.72:0/3720534418 conn(0x7f89e80fe970 msgr2=0x7f89e8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.809+0000 7f89ece00640 1 -- 172.21.15.72:0/3720534418 shutdown_connections 2024-03-11T19:22:57.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.809+0000 7f89ece00640 1 -- 172.21.15.72:0/3720534418 wait complete. 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 Processor -- start 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 -- start start 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 0x7f89e81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e8104f90 0x7f89e81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81a0e90 con 0x7f89e81045d0 2024-03-11T19:22:57.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81a1000 con 0x7f89e8104f90 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89ece00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81a11c0 con 0x7f89e81033e0 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89e5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 0x7f89e81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89e5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38896/0 (socket says 172.21.15.72:38896) 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.810+0000 7f89e5400640 1 -- 172.21.15.72:0/2971654514 learned_addr learned my addr 172.21.15.72:0/2971654514 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e6800640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e8104f90 0x7f89e81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 -- 172.21.15.72:0/2971654514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e8104f90 msgr2=0x7f89e81a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e8104f90 0x7f89e81a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 -- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 msgr2=0x7f89e81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 -- 172.21.15.72:0/2971654514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009560 con 0x7f89e81033e0 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5400640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:57.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.811+0000 7f89e5e00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 0x7f89e81993a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036040 tx=0x7f89d8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.812+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80043d0 con 0x7f89e81033e0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.812+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89d80052d0 con 0x7f89e81033e0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.812+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003cd0 con 0x7f89e81033e0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.812+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e81a1440 con 0x7f89e81033e0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.812+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e81a18d0 con 0x7f89e81033e0 2024-03-11T19:22:57.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.813+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e810a6a0 con 0x7f89e81033e0 2024-03-11T19:22:57.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.816+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f89d8005440 con 0x7f89e81033e0 2024-03-11T19:22:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.818+0000 7f89dea00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 0x7f89cc0a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.818+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f89d80efb10 con 0x7f89e81033e0 2024-03-11T19:22:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.818+0000 7f89e5400640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 0x7f89cc0a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:57.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.819+0000 7f89e5400640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 0x7f89cc0a3f90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89e819a8b0 tx=0x7f89d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:57.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:57.819+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89d80b8940 con 0x7f89e81033e0 2024-03-11T19:22:58.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.086+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f89e802ce40 con 0x7f89e81033e0 2024-03-11T19:22:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:22:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:22:58.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:58 smithi106 ceph-mon[35889]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:58.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:58 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2971654514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:58 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:22:58.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:22:58.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:58 smithi072 ceph-mon[31695]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:58.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:58 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2971654514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:58 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:22:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:22:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:58 smithi143 ceph-mon[36013]: from='mgr.14152 172.21.15.72:0/3173048416' entity='mgr.x' 2024-03-11T19:22:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:58 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2971654514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:58 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:22:58.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.719+0000 7f89dea00640 1 -- 172.21.15.72:0/2971654514 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89d80bc000 con 0x7f89e81033e0 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 msgr2=0x7f89cc0a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 0x7f89cc0a3f90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89e819a8b0 tx=0x7f89d4005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 msgr2=0x7f89e81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 0x7f89e81993a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036040 tx=0x7f89d8005e50 comp rx=0 tx=0).stop 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 shutdown_connections 2024-03-11T19:22:58.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f89cc0a1ad0 0x7f89cc0a3f90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89e8104f90 0x7f89e81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89e81045d0 0x7f89e81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 --2- 172.21.15.72:0/2971654514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e81033e0 0x7f89e81993a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 >> 172.21.15.72:0/2971654514 conn(0x7f89e80fe970 msgr2=0x7f89e8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 shutdown_connections 2024-03-11T19:22:58.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.722+0000 7f89ece00640 1 -- 172.21.15.72:0/2971654514 wait complete. 2024-03-11T19:22:58.744 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 -- 172.21.15.72:0/4195908312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 msgr2=0x7f85e8074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 --2- 172.21.15.72:0/4195908312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e8074060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009cd0 tx=0x7f85d002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 -- 172.21.15.72:0/4195908312 shutdown_connections 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 --2- 172.21.15.72:0/4195908312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e810d5c0 0x7f85e810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 --2- 172.21.15.72:0/4195908312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e8074060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 --2- 172.21.15.72:0/4195908312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e80751c0 0x7f85e8073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 -- 172.21.15.72:0/4195908312 >> 172.21.15.72:0/4195908312 conn(0x7f85e80fc560 msgr2=0x7f85e80fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 -- 172.21.15.72:0/4195908312 shutdown_connections 2024-03-11T19:22:58.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.959+0000 7f85eca00640 1 -- 172.21.15.72:0/4195908312 wait complete. 2024-03-11T19:22:58.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 Processor -- start 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 -- start start 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e80751c0 0x7f85e819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 0x7f85e81a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e81a5180 con 0x7f85e8073be0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e81a52f0 con 0x7f85e80751c0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85eca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e81a54b0 con 0x7f85e810d5c0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.961+0000 7f85e7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38910/0 (socket says 172.21.15.72:38910) 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 0x7f85e81a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e80751c0 0x7f85e819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7400640 1 -- 172.21.15.72:0/2028803880 learned_addr learned my addr 172.21.15.72:0/2028803880 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:58.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 -- 172.21.15.72:0/2028803880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e80751c0 msgr2=0x7f85e819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e80751c0 0x7f85e819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 -- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 msgr2=0x7f85e819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 -- 172.21.15.72:0/2028803880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009490 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7400640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.962+0000 7f85e7e00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 0x7f85e81a4c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e970 tx=0x7f85e000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.963+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000ccc0 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.963+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e000ce20 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.963+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d0009900 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.963+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0010430 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.963+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e81a5a90 con 0x7f85e810d5c0 2024-03-11T19:22:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.964+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8103680 con 0x7f85e810d5c0 2024-03-11T19:22:58.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.966+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f85e0004020 con 0x7f85e810d5c0 2024-03-11T19:22:58.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.967+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 0x7f85c80a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:58.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.967+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f85e0014070 con 0x7f85e810d5c0 2024-03-11T19:22:58.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.967+0000 7f85e7400640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 0x7f85c80a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:58.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.968+0000 7f85e7400640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 0x7f85c80a3ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f85d8011550 tx=0x7f85d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:58.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:58.971+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85e0096970 con 0x7f85e810d5c0 2024-03-11T19:22:59.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.197+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f85e802ceb0 con 0x7f85c80a1a20 2024-03-11T19:22:59.226 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.222+0000 7f85dfe00640 1 -- 172.21.15.72:0/2028803880 <== mgr.14152 v2:172.21.15.72:6800/537254334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f85e802ceb0 con 0x7f85c80a1a20 2024-03-11T19:22:59.226 INFO:teuthology.orchestra.run.smithi072.stdout:{} 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 msgr2=0x7f85c80a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 0x7f85c80a3ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f85d8011550 tx=0x7f85d8005f90 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 msgr2=0x7f85e81a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 0x7f85e81a4c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e970 tx=0x7f85e000ee30 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 shutdown_connections 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f85c80a1a20 0x7f85c80a3ee0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85e810d5c0 0x7f85e81a4c40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85e80751c0 0x7f85e819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 --2- 172.21.15.72:0/2028803880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e8073be0 0x7f85e819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.225+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 >> 172.21.15.72:0/2028803880 conn(0x7f85e80fc560 msgr2=0x7f85e8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:59.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.226+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 shutdown_connections 2024-03-11T19:22:59.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.226+0000 7f85eca00640 1 -- 172.21.15.72:0/2028803880 wait complete. 2024-03-11T19:22:59.247 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-11T19:22:59.258 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:22:59.300 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:22:59.300 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:22:59.300 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 -- 172.21.15.72:0/1184330098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520073b80 msgr2=0x7f5520073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 --2- 172.21.15.72:0/1184330098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520073b80 0x7f5520073fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f550c009900 tx=0x7f550c02bcd0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 -- 172.21.15.72:0/1184330098 shutdown_connections 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 --2- 172.21.15.72:0/1184330098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520108e80 0x7f552010b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 --2- 172.21.15.72:0/1184330098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520073b80 0x7f5520073fe0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 --2- 172.21.15.72:0/1184330098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 0x7f55200735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 -- 172.21.15.72:0/1184330098 >> 172.21.15.72:0/1184330098 conn(0x7f55200fc560 msgr2=0x7f55200fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:22:59.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.495+0000 7f551f200640 1 -- 172.21.15.72:0/1184330098 shutdown_connections 2024-03-11T19:22:59.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.496+0000 7f551f200640 1 -- 172.21.15.72:0/1184330098 wait complete. 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.496+0000 7f551f200640 1 Processor -- start 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.496+0000 7f551f200640 1 -- start start 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520073b80 0x7f5520199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 0x7f55201997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55201a0d50 con 0x7f5520108e80 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55201a0ec0 con 0x7f5520075160 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55201a1080 con 0x7f5520073b80 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520073b80 0x7f5520199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38948/0 (socket says 172.21.15.72:38948) 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 -- 172.21.15.72:0/1660766434 learned_addr learned my addr 172.21.15.72:0/1660766434 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551d400640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 0x7f55201997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 -- 172.21.15.72:0/1660766434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520073b80 msgr2=0x7f5520199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:59.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520073b80 0x7f5520199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 -- 172.21.15.72:0/1660766434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 msgr2=0x7f55201997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 0x7f55201997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.497+0000 7f551e800640 1 -- 172.21.15.72:0/1660766434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c009560 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f551e800640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f551800eb10 tx=0x7f551800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518004b20 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5518004c80 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518010650 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55201a1360 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.498+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55201a17f0 con 0x7f5520108e80 2024-03-11T19:22:59.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.499+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520107c40 con 0x7f5520108e80 2024-03-11T19:22:59.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.501+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5518005020 con 0x7f5520108e80 2024-03-11T19:22:59.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.503+0000 7f5516a00640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 0x7f55040a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:22:59.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.503+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f5518014070 con 0x7f5520108e80 2024-03-11T19:22:59.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.503+0000 7f551de00640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 0x7f55040a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:22:59.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.504+0000 7f551de00640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 0x7f55040a3ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5510009bd0 tx=0x7f5510005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:22:59.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.508+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5518096b60 con 0x7f5520108e80 2024-03-11T19:22:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:59 smithi106 ceph-mon[35889]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:22:59.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:59 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:22:59.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:22:59 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:22:59.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:22:59.208+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:22:59.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:22:59.209+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:22:59.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:22:59.209+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:22:59.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:22:59.209+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:22:59.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:22:59.209+0000 7f1ee2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:22:59.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-mon[31695]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:22:59.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:22:59.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:22:59 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:22:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:59 smithi143 ceph-mon[36013]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:22:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:59 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:22:59.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:22:59 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:22:59.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:22:59.759+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f552010ec50 con 0x7f5520108e80 2024-03-11T19:23:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.321+0000 7f5516a00640 1 -- 172.21.15.72:0/1660766434 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f552010ec50 con 0x7f5520108e80 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 msgr2=0x7f55040a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 0x7f55040a3ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5510009bd0 tx=0x7f5510005f90 comp rx=0 tx=0).stop 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 msgr2=0x7f55201a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f551800eb10 tx=0x7f551800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 shutdown_connections 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55040a1a20 0x7f55040a3ee0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5520108e80 0x7f55201a0810 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5520075160 0x7f55201997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 --2- 172.21.15.72:0/1660766434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5520073b80 0x7f5520199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 >> 172.21.15.72:0/1660766434 conn(0x7f55200fc560 msgr2=0x7f5520104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:00.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 shutdown_connections 2024-03-11T19:23:00.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.325+0000 7f551f200640 1 -- 172.21.15.72:0/1660766434 wait complete. 2024-03-11T19:23:00.355 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-11T19:23:00.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 -- 172.21.15.72:0/370882033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b14102ba0 msgr2=0x7f2b14109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 --2- 172.21.15.72:0/370882033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b14102ba0 0x7f2b14109430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009ca0 tx=0x7f2b0c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 -- 172.21.15.72:0/370882033 shutdown_connections 2024-03-11T19:23:00.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 --2- 172.21.15.72:0/370882033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b14102ba0 0x7f2b14109430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 --2- 172.21.15.72:0/370882033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b141021e0 0x7f2b14102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 --2- 172.21.15.72:0/370882033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14100ff0 0x7f2b14101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 -- 172.21.15.72:0/370882033 >> 172.21.15.72:0/370882033 conn(0x7f2b140fc560 msgr2=0x7f2b140fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 -- 172.21.15.72:0/370882033 shutdown_connections 2024-03-11T19:23:00.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.542+0000 7f2b13200640 1 -- 172.21.15.72:0/370882033 wait complete. 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 Processor -- start 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 -- start start 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b14100ff0 0x7f2b14074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14102ba0 0x7f2b14073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14074210 con 0x7f2b14102ba0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14074380 con 0x7f2b14100ff0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b13200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141a10a0 con 0x7f2b141021e0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b0be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.543+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b14100ff0 0x7f2b14074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b12800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14102ba0 0x7f2b14073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:49354/0 (socket says 172.21.15.72:49354) 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 -- 172.21.15.72:0/1717013274 learned_addr learned my addr 172.21.15.72:0/1717013274 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b14100ff0 msgr2=0x7f2b14074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b14100ff0 0x7f2b14074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14102ba0 msgr2=0x7f2b14073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14102ba0 0x7f2b14073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 -- 172.21.15.72:0/1717013274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc009490 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0be00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00ea60 tx=0x7f2afc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00cde0 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2afc0035f0 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc010670 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.544+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0c009900 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.545+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b141a1680 con 0x7f2b141021e0 2024-03-11T19:23:00.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.546+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b141082b0 con 0x7f2b141021e0 2024-03-11T19:23:00.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.547+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f2afc003af0 con 0x7f2b141021e0 2024-03-11T19:23:00.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.549+0000 7f2b0aa00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 0x7f2af80a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.549+0000 7f2b11e00640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 msgr2=0x7f2af80a3fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:00.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.549+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f2afc014070 con 0x7f2b141021e0 2024-03-11T19:23:00.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.549+0000 7f2b11e00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 0x7f2af80a3fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:23:00.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.553+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2afc096a50 con 0x7f2b141021e0 2024-03-11T19:23:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-mon[35889]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:00.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:23:00.646 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: ignoring --setuser ceph since I am not root 2024-03-11T19:23:00.646 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: ignoring --setgroup ceph since I am not root 2024-03-11T19:23:00.646 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-mgr[37282]: -- 172.21.15.106:0/1609877151 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b1f6ec1a0 con 0x564b1f6c2c00 2024-03-11T19:23:00.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:00 smithi072 ceph-mon[31695]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:00.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:23:00.682 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setuser ceph since I am not root 2024-03-11T19:23:00.682 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: ignoring --setgroup ceph since I am not root 2024-03-11T19:23:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-mon[36013]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:23:00.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: ignoring --setuser ceph since I am not root 2024-03-11T19:23:00.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: ignoring --setgroup ceph since I am not root 2024-03-11T19:23:00.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-mgr[37738]: -- 172.21.15.143:0/527276237 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55865603a1a0 con 0x558656010800 2024-03-11T19:23:00.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.750+0000 7f2b11e00640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 msgr2=0x7f2af80a3fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:00.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.750+0000 7f2b11e00640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 0x7f2af80a3fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:23:00.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.757+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f2b1410c9d0 con 0x7f2b141021e0 2024-03-11T19:23:00.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.767+0000 7f2b0aa00640 1 -- 172.21.15.72:0/1717013274 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2afc09a110 con 0x7f2b141021e0 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 msgr2=0x7f2af80a3fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 0x7f2af80a3fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 msgr2=0x7f2b14073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00ea60 tx=0x7f2afc00ef20 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 shutdown_connections 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f2af80a1af0 0x7f2af80a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b14102ba0 0x7f2b14073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b141021e0 0x7f2b14073570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 --2- 172.21.15.72:0/1717013274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b14100ff0 0x7f2b14074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.769+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 >> 172.21.15.72:0/1717013274 conn(0x7f2b140fc560 msgr2=0x7f2b14105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.770+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 shutdown_connections 2024-03-11T19:23:00.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.770+0000 7f2b13200640 1 -- 172.21.15.72:0/1717013274 wait complete. 2024-03-11T19:23:00.791 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-11T19:23:00.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 -- 172.21.15.72:0/417531222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881033e0 msgr2=0x7f5788103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 --2- 172.21.15.72:0/417531222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881033e0 0x7f5788103800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 -- 172.21.15.72:0/417531222 shutdown_connections 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 --2- 172.21.15.72:0/417531222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f578810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 --2- 172.21.15.72:0/417531222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881045d0 0x7f5788104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 --2- 172.21.15.72:0/417531222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881033e0 0x7f5788103800 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 -- 172.21.15.72:0/417531222 >> 172.21.15.72:0/417531222 conn(0x7f57880fe970 msgr2=0x7f5788100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 -- 172.21.15.72:0/417531222 shutdown_connections 2024-03-11T19:23:00.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.978+0000 7f578f400640 1 -- 172.21.15.72:0/417531222 wait complete. 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 Processor -- start 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 -- start start 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 0x7f57881993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f57881a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a0e90 con 0x7f5788104f90 2024-03-11T19:23:00.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a1000 con 0x7f57881033e0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.979+0000 7f578f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a11c0 con 0x7f57881045d0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f578d400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f57881a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f578ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 0x7f57881993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:49368/0 (socket says 172.21.15.72:49368) 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 -- 172.21.15.72:0/2178805350 learned_addr learned my addr 172.21.15.72:0/2178805350 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 msgr2=0x7f57881993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 0x7f57881993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 msgr2=0x7f57881a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f57881a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 -- 172.21.15.72:0/2178805350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c009560 con 0x7f57881045d0 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f578ca00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 0x7f57881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f578d400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f57881a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5787e00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f577400e9f0 tx=0x7f577400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.980+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774009720 con 0x7f57881045d0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.981+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f577400cd80 con 0x7f57881045d0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.981+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57881a14a0 con 0x7f57881045d0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.981+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774010690 con 0x7f57881045d0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.981+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57881a19b0 con 0x7f57881045d0 2024-03-11T19:23:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.982+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578810a6a0 con 0x7f57881045d0 2024-03-11T19:23:00.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.984+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 136296+0+0 (secure 0 0 0) 0x7f57740046c0 con 0x7f57881045d0 2024-03-11T19:23:00.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.985+0000 7f5785600640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 0x7f57700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:00.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.985+0000 7f578ca00640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 msgr2=0x7f57700a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:00.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.985+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f5774014070 con 0x7f57881045d0 2024-03-11T19:23:00.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.985+0000 7f578ca00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 0x7f57700a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:23:00.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:00.989+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5774096970 con 0x7f57881045d0 2024-03-11T19:23:01.145 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:00.886+0000 7f5ee91aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:23:01.145 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:00.978+0000 7f5ee91aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:23:01.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.184+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f578810edf0 con 0x7f57881045d0 2024-03-11T19:23:01.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:00.892+0000 7f536cbc3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:23:01.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:00.985+0000 7f536cbc3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:23:01.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.186+0000 7f578ca00640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 msgr2=0x7f57700a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:01.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.186+0000 7f578ca00640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 0x7f57700a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:23:01.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.192+0000 7f5785600640 1 -- 172.21.15.72:0/2178805350 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f577409a030 con 0x7f57881045d0 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 msgr2=0x7f57700a4060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 0x7f57700a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 msgr2=0x7f57881998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f577400e9f0 tx=0x7f577400edf0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 shutdown_connections 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f57700a1ba0 0x7f57700a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5788104f90 0x7f57881a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57881045d0 0x7f57881998e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 --2- 172.21.15.72:0/2178805350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57881033e0 0x7f57881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 >> 172.21.15.72:0/2178805350 conn(0x7f57880fe970 msgr2=0x7f5788107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 shutdown_connections 2024-03-11T19:23:01.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.195+0000 7f578f400640 1 -- 172.21.15.72:0/2178805350 wait complete. 2024-03-11T19:23:01.216 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 -- 172.21.15.72:0/929323780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5588100b40 msgr2=0x7f5588100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 --2- 172.21.15.72:0/929323780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5588100b40 0x7f5588100fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5580009ca0 tx=0x7f558002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 -- 172.21.15.72:0/929323780 shutdown_connections 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 --2- 172.21.15.72:0/929323780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5588100b40 0x7f5588100fc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 --2- 172.21.15.72:0/929323780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55881001a0 0x7f5588100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 --2- 172.21.15.72:0/929323780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55880ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 -- 172.21.15.72:0/929323780 >> 172.21.15.72:0/929323780 conn(0x7f55880fb460 msgr2=0x7f55880fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 -- 172.21.15.72:0/929323780 shutdown_connections 2024-03-11T19:23:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.419+0000 7f558ee00640 1 -- 172.21.15.72:0/929323780 wait complete. 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.420+0000 7f558ee00640 1 Processor -- start 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.420+0000 7f558ee00640 1 -- start start 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.420+0000 7f558ee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55881991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.420+0000 7f558ee00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55881001a0 0x7f5588199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.420+0000 7f558ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5588100b40 0x7f55881a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558ee00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881a0cd0 con 0x7f55881001a0 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558ee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881a0e40 con 0x7f55880ff840 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881a1000 con 0x7f5588100b40 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55881991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55881001a0 0x7f5588199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55881991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:49376/0 (socket says 172.21.15.72:49376) 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 -- 172.21.15.72:0/928528046 learned_addr learned my addr 172.21.15.72:0/928528046 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558e400640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5588100b40 0x7f55881a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5588100b40 msgr2=0x7f55881a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5588100b40 0x7f55881a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55881001a0 msgr2=0x7f5588199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55881001a0 0x7f5588199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c009490 con 0x7f55880ff840 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55881001a0 0x7f5588199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.421+0000 7f558da00640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55881991e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f557c00ea60 tx=0x7f557c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.422+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00cde0 con 0x7f55880ff840 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.422+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5580009900 con 0x7f55880ff840 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.422+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f557c0035f0 con 0x7f55880ff840 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.422+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c010690 con 0x7f55880ff840 2024-03-11T19:23:01.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.422+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55881a16b0 con 0x7f55880ff840 2024-03-11T19:23:01.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.423+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558810a6d0 con 0x7f55880ff840 2024-03-11T19:23:01.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.425+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 20) v1 ==== 136347+0+0 (secure 0 0 0) 0x7f557c003af0 con 0x7f55880ff840 2024-03-11T19:23:01.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.426+0000 7f5586000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:01.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.426+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:01.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.426+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f557c014070 con 0x7f55880ff840 2024-03-11T19:23:01.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.426+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:23:01.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.430+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f557c096b10 con 0x7f55880ff840 2024-03-11T19:23:01.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-mon[35889]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:23:01.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:23:01.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-mon[35889]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:23:01.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.627+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:01.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.627+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:23:01.634 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-mon[36013]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:23:01.634 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:23:01.634 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-mon[36013]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:23:01.634 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.634 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:01.648+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f558802ceb0 con 0x7f55700a1b00 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-mon[31695]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1660766434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-mon[31695]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:01.192+0000 7fd397cf0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:23:01.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:01.322+0000 7fd397cf0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:23:01.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:01.619+0000 7f5ee91aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:23:01.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:01.711+0000 7f5ee91aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:23:01.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:01.849+0000 7f5ee91aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:23:01.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:01.629+0000 7f536cbc3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:23:01.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:01.721+0000 7f536cbc3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:23:01.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:01.863+0000 7f536cbc3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:23:02.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:02.028+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:02.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:02.028+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:23:02.291 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:01.946+0000 7f5ee91aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:23:02.291 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.028+0000 7f5ee91aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:23:02.291 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.286+0000 7f5ee91aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:23:02.311 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:01.959+0000 7f536cbc3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:23:02.311 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.041+0000 7f536cbc3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:23:02.311 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.306+0000 7f536cbc3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:23:02.422 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-mon[31695]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-11T19:23:02.423 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.118+0000 7fd397cf0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:23:02.423 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.239+0000 7fd397cf0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:23:02.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-mon[35889]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-11T19:23:02.583 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.399+0000 7f5ee91aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:02.583 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.485+0000 7f5ee91aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:23:02.583 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.578+0000 7f5ee91aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:23:02.604 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-mon[36013]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-03-11T19:23:02.604 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.420+0000 7f536cbc3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:02.604 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.505+0000 7f536cbc3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:23:02.604 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.599+0000 7f536cbc3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:23:02.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.417+0000 7fd397cf0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:23:02.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.541+0000 7fd397cf0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:23:02.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.652+0000 7fd397cf0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:23:02.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:02.829+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:02.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:02.829+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:23:02.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:02.751+0000 7f5ee91aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:23:02.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:02.777+0000 7f536cbc3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:23:03.238 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:02.982+0000 7fd397cf0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:23:03.239 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:03.122+0000 7fd397cf0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:03.582 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:03.233+0000 7fd397cf0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:23:03.582 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:03.356+0000 7fd397cf0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:23:03.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:03.577+0000 7fd397cf0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:23:04.145 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:03.703+0000 7f5ee91aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:23:04.145 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:03.831+0000 7f5ee91aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:23:04.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:03.735+0000 7f536cbc3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:23:04.189 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:03.862+0000 7f536cbc3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:23:04.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:04.431+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:04.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:04.431+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:23:04.442 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:04.158+0000 7f5ee91aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:23:04.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:04.246+0000 7f5ee91aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:23:04.483 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:04.195+0000 7f536cbc3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:23:04.484 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:04.285+0000 7f536cbc3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:23:04.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:04.438+0000 7f5ee91aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:04.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:04.619+0000 7f5ee91aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:23:04.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:04.479+0000 7f536cbc3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:04.939 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:04.662+0000 7f536cbc3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:23:05.156 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:04.896+0000 7f5ee91aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:05.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:04.850+0000 7fd397cf0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:23:05.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:05.023+0000 7fd397cf0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:23:05.209 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:04.942+0000 7f536cbc3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:05.645 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:05.151+0000 7f5ee91aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:05.645 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:05.385+0000 7f5ee91aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:23:05.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:05.204+0000 7f536cbc3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:05.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:05.440+0000 7f536cbc3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:23:05.823 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:05.441+0000 7fd397cf0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:23:05.823 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:05.561+0000 7fd397cf0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:23:06.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:05.817+0000 7fd397cf0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:06.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:06.060+0000 7fd397cf0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:23:06.645 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:06.175+0000 7f5ee91aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:23:06.689 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:06.232+0000 7f536cbc3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:23:06.761 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:06.437+0000 7fd397cf0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:23:07.056 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:06.738+0000 7f5ee91aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:23:07.056 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:06.838+0000 7f5ee91aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:23:07.056 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y[37278]: 2024-03-11T19:23:07.051+0000 7f5ee91aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:23:07.073 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:06.756+0000 7fd397cf0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:23:07.121 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:06.805+0000 7f536cbc3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:23:07.121 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:06.903+0000 7f536cbc3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:23:07.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:07.136+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 21) v1 ==== 136347+0+0 (secure 0 0 0) 0x7f557c096ee0 con 0x7f55880ff840 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: Standby manager daemon y restarted 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: Standby manager daemon y started 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:23:07.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:07 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:07.067+0000 7fd397cf0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: Standby manager daemon y restarted 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: Standby manager daemon y started 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:07.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:23:07.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:07 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:07.439 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z[37694]: 2024-03-11T19:23:07.118+0000 7f536cbc3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:23:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: Standby manager daemon y restarted 2024-03-11T19:23:07.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:23:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: Standby manager daemon y started 2024-03-11T19:23:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:23:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:07 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.106:0/1760343758' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:07.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:07.635+0000 7f558d000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/537254334 2024-03-11T19:23:07.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:07.635+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-11T19:23:08.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:08.142+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 8 ==== mgrmap(e 22) v1 ==== 136347+0+0 (secure 0 0 0) 0x7f557c0cb660 con 0x7f55880ff840 2024-03-11T19:23:08.363 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:08.098+0000 7fd397cf0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:23:08.363 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: Standby manager daemon z restarted 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: Standby manager daemon z started 2024-03-11T19:23:08.364 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-mon[31695]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-11T19:23:08.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:23:08.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: Standby manager daemon z restarted 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: Standby manager daemon z started 2024-03-11T19:23:08.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:08 smithi143 ceph-mon[36013]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-11T19:23:08.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:23:08.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: mgrmap e21: x(active, since 11m), standbys: z, y 2024-03-11T19:23:08.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:23:08.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:23:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: from='mgr.? 172.21.15.143:0/2456123874' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:23:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: Standby manager daemon z restarted 2024-03-11T19:23:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: Standby manager daemon z started 2024-03-11T19:23:08.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:08 smithi106 ceph-mon[35889]: mgrmap e22: x(active, since 11m), standbys: y, z 2024-03-11T19:23:09.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:08.812+0000 7fd397cf0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:23:09.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:08.945+0000 7fd397cf0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:23:09.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:09.263+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 9 ==== mgrmap(e 23) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f557c003760 con 0x7f55880ff840 2024-03-11T19:23:09.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:09.265+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 msgr2=0x7f55700a3fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:23:09.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:09.265+0000 7f5586000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/537254334,v1:172.21.15.72:6801/537254334] conn(0x7f55700a1b00 0x7f55700a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: Active manager daemon x restarted 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: Activating manager daemon x 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: mgrmap e23: x(active, starting, since 0.0291648s), standbys: y, z 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:23:09.556 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:23:09.557 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:09 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:23:09.558 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: Active manager daemon x restarted 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: Activating manager daemon x 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: mgrmap e23: x(active, starting, since 0.0291648s), standbys: y, z 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:23:09.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:23:09.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:23:09.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:23:09.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:23:09.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:23:09.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:23:09.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:09 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:23:09.593 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.236+0000 7fd397cf0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: Active manager daemon x restarted 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: Activating manager daemon x 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: mgrmap e23: x(active, starting, since 0.0291648s), standbys: y, z 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:23:09.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:23:09.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:23:09.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:23:09.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ea600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ea600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ea600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ea600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ea600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ee200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ee200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ee200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ee200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:09.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:23:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:23:09.588+0000 7fd2ee200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:23:10.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.332+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mon.2 v2:172.21.15.106:3300/0 10 ==== mgrmap(e 24) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f557c0977b0 con 0x7f55880ff840 2024-03-11T19:23:10.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.334+0000 7f5586000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f557004c970 0x7f557004cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:10.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.334+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f558802ceb0 con 0x7f557004c970 2024-03-11T19:23:10.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.343+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f557004c970 0x7f557004cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:10.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.351+0000 7f558d000640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f557004c970 0x7f557004cd50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f558819a6f0 tx=0x7f5574005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.436+0000 7f5586000640 1 -- 172.21.15.72:0/928528046 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f558802ceb0 con 0x7f557004c970 2024-03-11T19:23:10.441 INFO:teuthology.orchestra.run.smithi072.stdout:Schedule set for path / 2024-03-11T19:23:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f557004c970 msgr2=0x7f557004cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f557004c970 0x7f557004cd50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f558819a6f0 tx=0x7f5574005f90 comp rx=0 tx=0).stop 2024-03-11T19:23:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 msgr2=0x7f55881991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 --2- 172.21.15.72:0/928528046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55880ff840 0x7f55881991e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f557c00ea60 tx=0x7f557c00ef20 comp rx=0 tx=0).stop 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558e400640 1 -- 172.21.15.72:0/928528046 reap_dead start 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 shutdown_connections 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 >> 172.21.15.72:0/928528046 conn(0x7f55880fb460 msgr2=0x7f55880fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 shutdown_connections 2024-03-11T19:23:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.439+0000 7f558ee00640 1 -- 172.21.15.72:0/928528046 wait complete. 2024-03-11T19:23:10.466 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: Manager daemon x is now available 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:23:10.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:10 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.645+0000 7f5cbca00640 1 -- 172.21.15.72:0/2450384112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81033e0 msgr2=0x7f5cb8103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.645+0000 7f5cbca00640 1 --2- 172.21.15.72:0/2450384112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81033e0 0x7f5cb8103800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009900 tx=0x7f5ca802bc50 comp rx=0 tx=0).stop 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 -- 172.21.15.72:0/2450384112 shutdown_connections 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 --2- 172.21.15.72:0/2450384112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 0x7f5cb810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 --2- 172.21.15.72:0/2450384112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81045d0 0x7f5cb8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 --2- 172.21.15.72:0/2450384112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81033e0 0x7f5cb8103800 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 -- 172.21.15.72:0/2450384112 >> 172.21.15.72:0/2450384112 conn(0x7f5cb80fe970 msgr2=0x7f5cb8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 -- 172.21.15.72:0/2450384112 shutdown_connections 2024-03-11T19:23:10.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 -- 172.21.15.72:0/2450384112 wait complete. 2024-03-11T19:23:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.646+0000 7f5cbca00640 1 Processor -- start 2024-03-11T19:23:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 -- start start 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 0x7f5cb8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 0x7f5cb81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81a0eb0 con 0x7f5cb81033e0 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81a1020 con 0x7f5cb81045d0 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cbca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81a11e0 con 0x7f5cb8104f90 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47200/0 (socket says 172.21.15.72:47200) 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cb5e00640 1 -- 172.21.15.72:0/295690987 learned_addr learned my addr 172.21.15.72:0/295690987 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cb5400640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 0x7f5cb8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.647+0000 7f5cb6800640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 0x7f5cb81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 -- 172.21.15.72:0/295690987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 msgr2=0x7f5cb81a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 0x7f5cb81a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 -- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 msgr2=0x7f5cb81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 -- 172.21.15.72:0/295690987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8009560 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5e00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cb5400640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 0x7f5cb8199900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9f0 tx=0x7f5ca400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400ccc0 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ca400ce20 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb81a14c0 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018920 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.648+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb81a19d0 con 0x7f5cb81045d0 2024-03-11T19:23:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.649+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb810a6a0 con 0x7f5cb81045d0 2024-03-11T19:23:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.652+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 24) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f5ca4018a80 con 0x7f5cb81045d0 2024-03-11T19:23:10.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.653+0000 7f5caea00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 0x7f5c9c0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:10.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.653+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f5ca4014070 con 0x7f5cb81045d0 2024-03-11T19:23:10.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.653+0000 7f5cb5e00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 0x7f5c9c0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:10.658 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.654+0000 7f5cb5e00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 0x7f5c9c0a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ca802c860 tx=0x7f5ca802cd30 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:10.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.657+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ca4052d20 con 0x7f5cb81045d0 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: Manager daemon x is now available 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:23:10.664 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:10 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: Manager daemon x is now available 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:23:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:10 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:23:10.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.883+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cb802ceb0 con 0x7f5c9c0a18e0 2024-03-11T19:23:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.982+0000 7f5caea00640 1 -- 172.21.15.72:0/295690987 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5cb802ceb0 con 0x7f5c9c0a18e0 2024-03-11T19:23:10.986 INFO:teuthology.orchestra.run.smithi072.stdout:Retention added to path / 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 msgr2=0x7f5c9c0a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 0x7f5c9c0a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ca802c860 tx=0x7f5ca802cd30 comp rx=0 tx=0).stop 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 msgr2=0x7f5cb8199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 0x7f5cb8199900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9f0 tx=0x7f5ca400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 shutdown_connections 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5c9c0a18e0 0x7f5c9c0a3da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb8104f90 0x7f5cb81a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cb81045d0 0x7f5cb8199900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 --2- 172.21.15.72:0/295690987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb81033e0 0x7f5cb81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.984+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 >> 172.21.15.72:0/295690987 conn(0x7f5cb80fe970 msgr2=0x7f5cb8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.985+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 shutdown_connections 2024-03-11T19:23:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:10.985+0000 7f5cbca00640 1 -- 172.21.15.72:0/295690987 wait complete. 2024-03-11T19:23:11.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-11T19:23:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 -- 172.21.15.72:0/3996627745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781043f0 msgr2=0x7fe678104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 --2- 172.21.15.72:0/3996627745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781043f0 0x7fe678104870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe660009900 tx=0x7fe66002bc30 comp rx=0 tx=0).stop 2024-03-11T19:23:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 -- 172.21.15.72:0/3996627745 shutdown_connections 2024-03-11T19:23:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 --2- 172.21.15.72:0/3996627745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 0x7fe67810b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 --2- 172.21.15.72:0/3996627745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781043f0 0x7fe678104870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 --2- 172.21.15.72:0/3996627745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe678103200 0x7fe678103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 -- 172.21.15.72:0/3996627745 >> 172.21.15.72:0/3996627745 conn(0x7fe6780fe790 msgr2=0x7fe678100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 -- 172.21.15.72:0/3996627745 shutdown_connections 2024-03-11T19:23:11.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.187+0000 7fe677200640 1 -- 172.21.15.72:0/3996627745 wait complete. 2024-03-11T19:23:11.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.188+0000 7fe677200640 1 Processor -- start 2024-03-11T19:23:11.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.188+0000 7fe677200640 1 -- start start 2024-03-11T19:23:11.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.188+0000 7fe677200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe677200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6781043f0 0x7fe678199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe677200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 0x7fe6781a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe677200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a0d40 con 0x7fe678104db0 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe677200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a0eb0 con 0x7fe678103200 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe677200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a1070 con 0x7fe6781043f0 2024-03-11T19:23:11.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56542/0 (socket says 172.21.15.72:56542) 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 -- 172.21.15.72:0/1380135249 learned_addr learned my addr 172.21.15.72:0/1380135249 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe676800640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 0x7fe6781a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe66fe00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6781043f0 0x7fe678199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 -- 172.21.15.72:0/1380135249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6781043f0 msgr2=0x7fe678199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6781043f0 0x7fe678199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 -- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 msgr2=0x7fe6781a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 0x7fe6781a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.189+0000 7fe675e00640 1 -- 172.21.15.72:0/1380135249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660009560 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe675e00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9c0 tx=0x7fe66800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6680096e0 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66800cd80 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6781a1350 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668010430 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.190+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a17e0 con 0x7fe678103200 2024-03-11T19:23:11.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.191+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67810a4c0 con 0x7fe678103200 2024-03-11T19:23:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.193+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 24) v1 ==== 136091+0+0 (secure 0 0 0) 0x7fe668003680 con 0x7fe678103200 2024-03-11T19:23:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.194+0000 7fe66ea00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 0x7fe65c0a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.194+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fe668014070 con 0x7fe678103200 2024-03-11T19:23:11.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.194+0000 7fe66fe00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 0x7fe65c0a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.195+0000 7fe66fe00640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 0x7fe65c0a3cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe660000c00 tx=0x7fe66002c7e0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:11.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.198+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe668096890 con 0x7fe678103200 2024-03-11T19:23:11.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.344+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 25) v1 ==== 136399+0+0 (secure 0 0 0) 0x7fe668096c60 con 0x7fe678103200 2024-03-11T19:23:11.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.432+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe67802cee0 con 0x7fe65c0a1810 2024-03-11T19:23:11.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.444+0000 7fe66ea00640 1 -- 172.21.15.72:0/1380135249 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fe67802cee0 con 0x7fe65c0a1810 2024-03-11T19:23:11.447 INFO:teuthology.orchestra.run.smithi072.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-11T00:00:00", "created": "2024-03-11T19:23:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 msgr2=0x7fe65c0a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 0x7fe65c0a3cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe660000c00 tx=0x7fe66002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 msgr2=0x7fe678199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9c0 tx=0x7fe66800ee80 comp rx=0 tx=0).stop 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 shutdown_connections 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe65c0a1810 0x7fe65c0a3cd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe678104db0 0x7fe6781a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6781043f0 0x7fe678199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 --2- 172.21.15.72:0/1380135249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe678103200 0x7fe678199250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.446+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 >> 172.21.15.72:0/1380135249 conn(0x7fe6780fe790 msgr2=0x7fe678107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.447+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 shutdown_connections 2024-03-11T19:23:11.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.447+0000 7fe677200640 1 -- 172.21.15.72:0/1380135249 wait complete. 2024-03-11T19:23:11.471 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-11T19:23:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: [11/Mar/2024:19:23:09] ENGINE Bus STARTING 2024-03-11T19:23:11.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: [11/Mar/2024:19:23:09] ENGINE Serving on https://172.21.15.72:7150 2024-03-11T19:23:11.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: [11/Mar/2024:19:23:10] ENGINE Serving on http://172.21.15.72:8765 2024-03-11T19:23:11.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: [11/Mar/2024:19:23:10] ENGINE Bus STARTED 2024-03-11T19:23:11.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: mgrmap e24: x(active, since 1.09484s), standbys: y, z 2024-03-11T19:23:11.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:11.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:11 smithi106 ceph-mon[35889]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 -- 172.21.15.72:0/2355802373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 msgr2=0x7fb4001015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 --2- 172.21.15.72:0/2355802373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001015e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009900 tx=0x7fb3f402bc30 comp rx=0 tx=0).stop 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 -- 172.21.15.72:0/2355802373 shutdown_connections 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 --2- 172.21.15.72:0/2355802373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb400102d70 0x7fb400109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 --2- 172.21.15.72:0/2355802373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4001023b0 0x7fb400102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 --2- 172.21.15.72:0/2355802373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001015e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 -- 172.21.15.72:0/2355802373 >> 172.21.15.72:0/2355802373 conn(0x7fb4000fc710 msgr2=0x7fb4000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 -- 172.21.15.72:0/2355802373 shutdown_connections 2024-03-11T19:23:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.661+0000 7fb408400640 1 -- 172.21.15.72:0/2355802373 wait complete. 2024-03-11T19:23:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.662+0000 7fb408400640 1 Processor -- start 2024-03-11T19:23:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.662+0000 7fb408400640 1 -- start start 2024-03-11T19:23:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.662+0000 7fb408400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb408400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4001023b0 0x7fb400197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb408400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb400102d70 0x7fb40019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb408400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40019ecc0 con 0x7fb4001023b0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb408400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40019ee30 con 0x7fb4001011c0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb408400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40019eff0 con 0x7fb400102d70 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4001023b0 0x7fb400197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56586/0 (socket says 172.21.15.72:56586) 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 -- 172.21.15.72:0/3365704315 learned_addr learned my addr 172.21.15.72:0/3365704315 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb406400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb400102d70 0x7fb40019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 -- 172.21.15.72:0/3365704315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb400102d70 msgr2=0x7fb40019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb400102d70 0x7fb40019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 -- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4001023b0 msgr2=0x7fb400197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4001023b0 0x7fb400197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.663+0000 7fb405a00640 1 -- 172.21.15.72:0/3365704315 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009560 con 0x7fb4001011c0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb405a00640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005ab0 tx=0x7fb3f4005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40043d0 con 0x7fb4001011c0 2024-03-11T19:23:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3f40052d0 con 0x7fb4001011c0 2024-03-11T19:23:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4003d40 con 0x7fb4001011c0 2024-03-11T19:23:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40019f270 con 0x7fb4001011c0 2024-03-11T19:23:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.664+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40019f780 con 0x7fb4001011c0 2024-03-11T19:23:11.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.665+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb400108500 con 0x7fb4001011c0 2024-03-11T19:23:11.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.667+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 25) v1 ==== 136399+0+0 (secure 0 0 0) 0x7fb3f4005440 con 0x7fb4001011c0 2024-03-11T19:23:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.668+0000 7fb3fe000640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 0x7fb3e80a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.668+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb3f40efc00 con 0x7fb4001011c0 2024-03-11T19:23:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.668+0000 7fb405000640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 0x7fb3e80a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:11.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.669+0000 7fb405000640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 0x7fb3e80a4100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4001986e0 tx=0x7fb3ec005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.672+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3f40b8800 con 0x7fb4001011c0 2024-03-11T19:23:11.679 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: [11/Mar/2024:19:23:09] ENGINE Bus STARTING 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: [11/Mar/2024:19:23:09] ENGINE Serving on https://172.21.15.72:7150 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: [11/Mar/2024:19:23:10] ENGINE Serving on http://172.21.15.72:8765 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: [11/Mar/2024:19:23:10] ENGINE Bus STARTED 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: mgrmap e24: x(active, since 1.09484s), standbys: y, z 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:11.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:11 smithi072 ceph-mon[31695]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: [11/Mar/2024:19:23:09] ENGINE Bus STARTING 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: [11/Mar/2024:19:23:09] ENGINE Serving on https://172.21.15.72:7150 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: [11/Mar/2024:19:23:10] ENGINE Serving on http://172.21.15.72:8765 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: [11/Mar/2024:19:23:10] ENGINE Bus STARTED 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: mgrmap e24: x(active, since 1.09484s), standbys: y, z 2024-03-11T19:23:11.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: from='client.25418 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:11.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:11 smithi143 ceph-mon[36013]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:11.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.913+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb40002ce40 con 0x7fb3e80a1c40 2024-03-11T19:23:11.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.922+0000 7fb3fe000640 1 -- 172.21.15.72:0/3365704315 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fb40002ce40 con 0x7fb3e80a1c40 2024-03-11T19:23:11.925 INFO:teuthology.orchestra.run.smithi072.stdout:/ 1m 6m3h 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 msgr2=0x7fb3e80a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 0x7fb3e80a4100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4001986e0 tx=0x7fb3ec005f90 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 msgr2=0x7fb4001971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005ab0 tx=0x7fb3f4005e50 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 shutdown_connections 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb3e80a1c40 0x7fb3e80a4100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb400102d70 0x7fb40019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4001023b0 0x7fb400197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 --2- 172.21.15.72:0/3365704315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4001011c0 0x7fb4001971d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.925+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 >> 172.21.15.72:0/3365704315 conn(0x7fb4000fc710 msgr2=0x7fb4001056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.926+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 shutdown_connections 2024-03-11T19:23:11.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:23:11.926+0000 7fb408400640 1 -- 172.21.15.72:0/3365704315 wait complete. 2024-03-11T19:23:11.949 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-11T19:23:12.042 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-11T19:23:12.053 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-11T19:23:12.092 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:23:12.092 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-11T19:23:12.097 INFO:teuthology.run_tasks:Running task workunit... 2024-03-11T19:23:12.128 INFO:tasks.workunit:Pulling workunits from ref b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T19:23:12.128 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-11T19:23:12.128 INFO:tasks.workunit:timeout=3h 2024-03-11T19:23:12.128 INFO:tasks.workunit:cleanup=True 2024-03-11T19:23:12.129 DEBUG:teuthology.orchestra.run.smithi072:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-11T19:23:12.171 INFO:teuthology.orchestra.run.smithi072.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-11T19:23:12.171 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-11T19:23:12.171 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-11T19:23:12.171 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:23:12.171 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-11T19:23:12.172 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-03-11 19:20:17.150678058 +0000 2024-03-11T19:23:12.172 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-03-11 19:20:17.150678058 +0000 2024-03-11T19:23:12.172 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-03-11 19:22:57.131656763 +0000 2024-03-11T19:23:12.172 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-03-11 19:20:17.150678058 +0000 2024-03-11T19:23:12.172 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-11T19:23:12.172 DEBUG:teuthology.orchestra.run.smithi072:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-11T19:23:12.295 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T19:23:12.345 INFO:tasks.workunit.client.0.smithi072.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-11T19:23:12.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:12 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:12 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: mgrmap e25: x(active, since 2s), standbys: y, z 2024-03-11T19:23:12.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='client.25466 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:12.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:12.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:12 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:13.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:13 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:13.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.682 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:13 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:23:13.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:13.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:13 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:14 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:23:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:14 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:14 smithi072 ceph-mon[31695]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:14.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:14 smithi072 ceph-mon[31695]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-11T19:23:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:14 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:23:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:14 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:14 smithi143 ceph-mon[36013]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:14 smithi143 ceph-mon[36013]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-11T19:23:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:14 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:23:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:14 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:23:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:14 smithi106 ceph-mon[35889]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:14 smithi106 ceph-mon[35889]: mgrmap e26: x(active, since 4s), standbys: y, z 2024-03-11T19:23:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:23:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:15 smithi106 ceph-mon[35889]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:23:15.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:15 smithi072 ceph-mon[31695]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:23:15.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: Updating smithi072:/etc/ceph/ceph.conf 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-11T19:23:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:15 smithi143 ceph-mon[36013]: Updating smithi143:/etc/ceph/ceph.conf 2024-03-11T19:23:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:16 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:16 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: Updating smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:16 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:17.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:17 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:17.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:17 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: Updating smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:23:17.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:23:17.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:23:17.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:17 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:23:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:18 smithi106 ceph-mon[35889]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-03-11T19:23:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:18 smithi072 ceph-mon[31695]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-03-11T19:23:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:18 smithi143 ceph-mon[36013]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 133 KiB/s wr, 78 op/s 2024-03-11T19:23:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:20 smithi106 ceph-mon[35889]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 60 op/s 2024-03-11T19:23:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:20 smithi072 ceph-mon[31695]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 60 op/s 2024-03-11T19:23:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:20 smithi143 ceph-mon[36013]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 60 op/s 2024-03-11T19:23:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:22 smithi106 ceph-mon[35889]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-03-11T19:23:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:22 smithi072 ceph-mon[31695]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-03-11T19:23:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:22 smithi143 ceph-mon[36013]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-03-11T19:23:24.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:24 smithi143 ceph-mon[36013]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:24.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:24 smithi106 ceph-mon[35889]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:24 smithi072 ceph-mon[31695]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 -- 172.21.15.72:0/3915385484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9908102da0 msgr2=0x7f990810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 --2- 172.21.15.72:0/3915385484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9908102da0 0x7f990810b1f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009900 tx=0x7f98f402bc30 comp rx=0 tx=0).stop 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 -- 172.21.15.72:0/3915385484 shutdown_connections 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 --2- 172.21.15.72:0/3915385484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f990810b900 0x7f990810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 --2- 172.21.15.72:0/3915385484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9908102da0 0x7f990810b1f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 --2- 172.21.15.72:0/3915385484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f99081023f0 0x7f99081027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.332+0000 7f990fa00640 1 -- 172.21.15.72:0/3915385484 >> 172.21.15.72:0/3915385484 conn(0x7f99080771a0 msgr2=0x7f99080775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.333+0000 7f990fa00640 1 -- 172.21.15.72:0/3915385484 shutdown_connections 2024-03-11T19:23:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.333+0000 7f990fa00640 1 -- 172.21.15.72:0/3915385484 wait complete. 2024-03-11T19:23:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 Processor -- start 2024-03-11T19:23:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 -- start start 2024-03-11T19:23:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99081023f0 0x7f9908199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 0x7f990819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819eee0 con 0x7f990810b900 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.334+0000 7f990fa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819f050 con 0x7f990819e4f0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990fa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990819f210 con 0x7f99081023f0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f9907e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 0x7f990819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:51202/0 (socket says 172.21.15.72:51202) 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f9907e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 0x7f990819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42978/0 (socket says 172.21.15.72:42978) 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 -- 172.21.15.72:0/1764149834 learned_addr learned my addr 172.21.15.72:0/1764149834 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990d000640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99081023f0 0x7f9908199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 -- 172.21.15.72:0/1764149834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99081023f0 msgr2=0x7f9908199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99081023f0 0x7f9908199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 -- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 msgr2=0x7f990819a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 0x7f990819a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.335+0000 7f990da00640 1 -- 172.21.15.72:0/1764149834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f4009560 con 0x7f990819e4f0 2024-03-11T19:23:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f990da00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f990000eb10 tx=0x7f990000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f9905600640 1 -- 172.21.15.72:0/1764149834 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900004b20 con 0x7f990819e4f0 2024-03-11T19:23:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99081a67a0 con 0x7f990819e4f0 2024-03-11T19:23:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f9905600640 1 -- 172.21.15.72:0/1764149834 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9900004c80 con 0x7f990819e4f0 2024-03-11T19:23:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f9905600640 1 -- 172.21.15.72:0/1764149834 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900010670 con 0x7f990819e4f0 2024-03-11T19:23:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.336+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99081a6c20 con 0x7f990819e4f0 2024-03-11T19:23:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.340+0000 7f9912813740 1 Processor -- start 2024-03-11T19:23:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.340+0000 7f9905600640 1 -- 172.21.15.72:0/1764149834 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f9900005020 con 0x7f990819e4f0 2024-03-11T19:23:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 -- start start 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x56446463b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464647720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644645f7620 con 0x56446463bdb0 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644645a98c0 con 0x56446463cb20 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f9912813740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644645a6df0 con 0x564464645330 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42994/0 (socket says 172.21.15.72:42994) 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.341+0000 7f990d000640 1 -- 172.21.15.72:0/4003411314 learned_addr learned my addr 172.21.15.72:0/4003411314 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f9907e00640 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x56446463b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 -- 172.21.15.72:0/4003411314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 msgr2=0x564464647720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464647720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 -- 172.21.15.72:0/4003411314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 msgr2=0x56446463b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x56446463b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 -- 172.21.15.72:0/4003411314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564464523b00 con 0x56446463bdb0 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f9905600640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 0x7f98f00a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f990d000640 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009cf0 tx=0x7f98fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0810465aac8c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f9905600640 1 -- 172.21.15.72:0/1764149834 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f9900014070 con 0x7f990819e4f0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.342+0000 7f9907e00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 0x7f98f00a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f98fa000640 1 -- 172.21.15.72:0/4003411314 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc009ea0 con 0x56446463bdb0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f98fa000640 1 -- 172.21.15.72:0/4003411314 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98fc004230 con 0x56446463bdb0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f98fa000640 1 -- 172.21.15.72:0/4003411314 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc019820 con 0x56446463bdb0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 -- 172.21.15.72:0/4003411314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 msgr2=0x56446463c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9907e00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 0x7f98f00a41a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4005e40 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009cf0 tx=0x7f98fc011810 comp rx=0 tx=0).stop 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 -- 172.21.15.72:0/4003411314 shutdown_connections 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446463c230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 --2- 172.21.15.72:0/4003411314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x56446463b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.343+0000 7f9912813740 1 -- 172.21.15.72:0/4003411314 >> 172.21.15.72:0/4003411314 conn(0x5644646034b0 msgr2=0x564464636aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.344+0000 7f9912813740 1 -- 172.21.15.72:0/4003411314 shutdown_connections 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.344+0000 7f9912813740 1 -- 172.21.15.72:0/4003411314 wait complete. 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.344+0000 7f9912813740 1 -- start start 2024-03-11T19:23:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.344+0000 7f9912813740 1 Processor -- start 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564464645330 0x564464642db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644645a25d0 con 0x564464645330 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464547b50 con 0x56446463cb20 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644645477f0 con 0x56446463bdb0 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9912813740 10 client.0 _ll_register_callbacks cb 0x56446455b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:50768/0 (socket says 172.21.15.72:50768) 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 -- 172.21.15.72:0/1512958460 learned_addr learned my addr 172.21.15.72:0/1512958460 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990da00640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564464645330 0x564464642db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9907e00640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 -- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 msgr2=0x564464642870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 -- 172.21.15.72:0/1512958460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564464645330 msgr2=0x564464642db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564464645330 0x564464642db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f990d000640 1 -- 172.21.15.72:0/1512958460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc009950 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.345+0000 7f9907e00640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f990d000640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009340 tx=0x7f98fc0052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a3dab8e24da645 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f98eea00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc0048c0 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564464574cd0 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f98fc004a20 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98fc009ea0 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f9912813740 10 client.25486 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.346+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5644645a0c50 con 0x56446463bdb0 2024-03-11T19:23:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564464574cd0 con 0x56446463bdb0 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f98fc021040 con 0x56446463bdb0 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 10 client.25486 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 20 client.25486 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564464655240 0x564464657630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 4 client.25486 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.347+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564464598ce0 con 0x564464655240 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.348+0000 7f9907e00640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564464655240 0x564464657630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.348+0000 7f9907e00640 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564464655240 0x564464657630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16be78d40b1beb5 server_cookie=c740d3d2c5254f38 in_seq=0 out_seq=0 2024-03-11T19:23:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.348+0000 7f98eea00640 10 client.25486 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f98eea00640 1 -- 172.21.15.72:0/1512958460 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564464598ce0 con 0x564464655240 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f98eea00640 10 client.25486 handle_command_reply: tid=0 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 client.25486 shutdown 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 msgr2=0x564464642330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009340 tx=0x7f98fc0052f0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 shutdown_connections 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564464655240 0x564464657630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564464645330 0x564464642db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f98eea00640 0 client.25486 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 --2- 172.21.15.72:0/1512958460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463bdb0 0x564464642330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.349+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 >> 172.21.15.72:0/1512958460 conn(0x5644646034b0 msgr2=0x5644646387f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.350+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 shutdown_connections 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.350+0000 7f9912813740 1 -- 172.21.15.72:0/1512958460 wait complete. 2024-03-11T19:23:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.350+0000 7f9912813740 20 client.25486 trim_cache size 0 max 16384 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 Processor -- start 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 -- start start 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464642dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644642eae60 con 0x56446463bdb0 2024-03-11T19:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644646327a0 con 0x56446463cb20 2024-03-11T19:23:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f9912813740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x564464651df0 con 0x564464645330 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.365+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43020/0 (socket says 172.21.15.72:43020) 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f990d000640 1 -- 172.21.15.72:0/2892153759 learned_addr learned my addr 172.21.15.72:0/2892153759 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f990da00640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464642dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 -- 172.21.15.72:0/2892153759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 msgr2=0x564464642dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464642dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 -- 172.21.15.72:0/2892153759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 msgr2=0x564464644a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 -- 172.21.15.72:0/2892153759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56446453d530 con 0x56446463cb20 2024-03-11T19:23:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f990d000640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f9907e00640 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c170 tx=0x7f98f4044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c97d171544a65c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f98f9600640 1 -- 172.21.15.72:0/2892153759 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4053ea0 con 0x56446463cb20 2024-03-11T19:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.366+0000 7f98f9600640 1 -- 172.21.15.72:0/2892153759 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98f40531d0 con 0x56446463cb20 2024-03-11T19:23:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f98f9600640 1 -- 172.21.15.72:0/2892153759 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4003970 con 0x56446463cb20 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 -- 172.21.15.72:0/2892153759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 msgr2=0x564464642880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c170 tx=0x7f98f4044000 comp rx=0 tx=0).stop 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 -- 172.21.15.72:0/2892153759 shutdown_connections 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564464645330 0x564464642dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56446463cb20 0x564464642880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 --2- 172.21.15.72:0/2892153759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x564464644a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 -- 172.21.15.72:0/2892153759 >> 172.21.15.72:0/2892153759 conn(0x5644646034b0 msgr2=0x564464640c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 -- 172.21.15.72:0/2892153759 shutdown_connections 2024-03-11T19:23:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.367+0000 7f9912813740 1 -- 172.21.15.72:0/2892153759 wait complete. 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 -- start start 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 Processor -- start 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 0x56446464a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 0x56446464af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x56446464b5e0 con 0x56446463bdb0 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x56446464b750 con 0x564464645330 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644646ab730 con 0x56446463cb20 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.368+0000 7f9912813740 10 client.0 _ll_register_callbacks cb 0x7f98fc009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43034/0 (socket says 172.21.15.72:43034) 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990d000640 1 -- 172.21.15.72:0/1754679522 learned_addr learned my addr 172.21.15.72:0/1754679522 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 0x56446464a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990da00640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 0x56446464af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 -- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 msgr2=0x56446464af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 0x56446464af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 -- 172.21.15.72:0/1754679522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 msgr2=0x56446464c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 -- 172.21.15.72:0/1754679522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644645a0c50 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990d000640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f990da00640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 0x56446464af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f9907e00640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 0x56446464a910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c800 tx=0x7f98f4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73683cd9b346e3a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.369+0000 7f98eea00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4053750 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644645c7d30 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f98f4048040 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f40538b0 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f9912813740 10 client.25492 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f990810ea10 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.370+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644645c7d30 con 0x56446463cb20 2024-03-11T19:23:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f98f4040970 con 0x56446463cb20 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 10 client.25492 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 20 client.25492 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5644646acbc0 0x5644646aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 4 client.25492 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5644642a2600 con 0x5644646acbc0 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f990d000640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5644646acbc0 0x5644646aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f990d000640 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5644646acbc0 0x5644646aefe0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135fc21219fda4ba server_cookie=c9e36647b91586be in_seq=0 out_seq=0 2024-03-11T19:23:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.371+0000 7f98eea00640 10 client.25492 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f98eea00640 1 -- 172.21.15.72:0/1754679522 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5644642a2600 con 0x5644646acbc0 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f98eea00640 10 client.25492 handle_command_reply: tid=0 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 client.25492 shutdown 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 msgr2=0x56446464a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 0x56446464a910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c800 tx=0x7f98f4004af0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 shutdown_connections 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5644646acbc0 0x5644646aefe0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564464645330 0x56446464af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56446463cb20 0x56446464a910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f9912813740 1 --2- 172.21.15.72:0/1754679522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56446463bdb0 0x56446464c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.383+0000 7f98eea00640 0 client.25492 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 >> 172.21.15.72:0/1754679522 conn(0x5644646034b0 msgr2=0x5644646354b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 shutdown_connections 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f9912813740 1 -- 172.21.15.72:0/1754679522 wait complete. 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f9912813740 20 client.25492 trim_cache size 0 max 16384 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:23:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 msgr2=0x7f98f00a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 0x7f98f00a41a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4005e40 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 msgr2=0x7f990819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.384+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f990000eb10 tx=0x7f990000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 shutdown_connections 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f98f00a1ce0 0x7f98f00a41a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f990819e4f0 0x7f990819e9a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f990810b900 0x7f990819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 --2- 172.21.15.72:0/1764149834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99081023f0 0x7f9908199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 >> 172.21.15.72:0/1764149834 conn(0x7f99080771a0 msgr2=0x7f9908107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 shutdown_connections 2024-03-11T19:23:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:25.385+0000 7f990fa00640 1 -- 172.21.15.72:0/1764149834 wait complete. 2024-03-11T19:23:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:26 smithi106 ceph-mon[35889]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:26 smithi072 ceph-mon[31695]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:26 smithi143 ceph-mon[36013]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:23:28.761 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:28 smithi072 ceph-mon[31695]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-11T19:23:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:28 smithi106 ceph-mon[35889]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-11T19:23:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:28 smithi143 ceph-mon[36013]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-11T19:23:30.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:30 smithi072 ceph-mon[31695]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:23:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:30 smithi106 ceph-mon[35889]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:23:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:30 smithi143 ceph-mon[36013]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:23:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:32 smithi106 ceph-mon[35889]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:32 smithi072 ceph-mon[31695]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:32 smithi143 ceph-mon[36013]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:34 smithi106 ceph-mon[35889]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:34 smithi072 ceph-mon[31695]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:34 smithi143 ceph-mon[36013]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:35.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:35 smithi106 ceph-mon[35889]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:35.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:35 smithi072 ceph-mon[31695]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:35 smithi143 ceph-mon[36013]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 -- 172.21.15.72:0/1903382833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc101b90 msgr2=0x7f11fc105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 --2- 172.21.15.72:0/1903382833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc101b90 0x7f11fc105bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009930 tx=0x7f11e802bc20 comp rx=0 tx=0).stop 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 -- 172.21.15.72:0/1903382833 shutdown_connections 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 --2- 172.21.15.72:0/1903382833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 --2- 172.21.15.72:0/1903382833 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc101b90 0x7f11fc105bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 --2- 172.21.15.72:0/1903382833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc1011e0 0x7f11fc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 -- 172.21.15.72:0/1903382833 >> 172.21.15.72:0/1903382833 conn(0x7f11fc0fcac0 msgr2=0x7f11fc0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 -- 172.21.15.72:0/1903382833 shutdown_connections 2024-03-11T19:23:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.528+0000 7f1200c00640 1 -- 172.21.15.72:0/1903382833 wait complete. 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.529+0000 7f1200c00640 1 Processor -- start 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.529+0000 7f1200c00640 1 -- start start 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 0x7f11fc19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc199660 0x7f11fc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19a130 con 0x7f11fc1011e0 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19a2a0 con 0x7f11fc199660 2024-03-11T19:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f1200c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19a460 con 0x7f11fc1062b0 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 0x7f11fc19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 0x7f11fc19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48430/0 (socket says 172.21.15.72:48430) 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11fb400640 1 -- 172.21.15.72:0/3456640717 learned_addr learned my addr 172.21.15.72:0/3456640717 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc199660 0x7f11fc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 -- 172.21.15.72:0/3456640717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc199660 msgr2=0x7f11fc199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc199660 0x7f11fc199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 -- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 msgr2=0x7f11fc19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 0x7f11fc19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.530+0000 7f11faa00640 1 -- 172.21.15.72:0/3456640717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec009490 con 0x7f11fc1062b0 2024-03-11T19:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.531+0000 7f11faa00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc19fae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009230 tx=0x7f11e802c5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.531+0000 7f11f3e00640 1 -- 172.21.15.72:0/3456640717 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e802dea0 con 0x7f11fc1062b0 2024-03-11T19:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.531+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e8009560 con 0x7f11fc1062b0 2024-03-11T19:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.531+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1a6740 con 0x7f11fc1062b0 2024-03-11T19:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.532+0000 7f11f3e00640 1 -- 172.21.15.72:0/3456640717 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e80042d0 con 0x7f11fc1062b0 2024-03-11T19:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.532+0000 7f11f3e00640 1 -- 172.21.15.72:0/3456640717 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8003830 con 0x7f11fc1062b0 2024-03-11T19:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.535+0000 7f11f3e00640 1 -- 172.21.15.72:0/3456640717 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f11e8003990 con 0x7f11fc1062b0 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.535+0000 7f120500d740 1 Processor -- start 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.535+0000 7f120500d740 1 -- start start 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.535+0000 7f120500d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f120500d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c2970ff0 0x5592c2971470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f120500d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c297a570 0x5592c297c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f120500d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c292c620 con 0x5592c297a570 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f120500d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c28de8c0 con 0x5592c2971d60 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f120500d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c28dbdf0 con 0x5592c2970ff0 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c297a570 0x5592c297c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:36770/0 (socket says 172.21.15.72:36770) 2024-03-11T19:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 -- 172.21.15.72:0/1282818487 learned_addr learned my addr 172.21.15.72:0/1282818487 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fb400640 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c2970ff0 0x5592c2971470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 -- 172.21.15.72:0/1282818487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c2970ff0 msgr2=0x5592c2971470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c2970ff0 0x5592c2971470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 -- 172.21.15.72:0/1282818487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c297a570 msgr2=0x5592c297c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c297a570 0x5592c297c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.536+0000 7f11fbe00640 1 -- 172.21.15.72:0/1282818487 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c2858b00 con 0x5592c2971d60 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11fbe00640 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009f50 tx=0x7f11f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73bfb66c8fc6833 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11f3e00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 0x7f11e00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11f3e00640 1 -- 172.21.15.72:0/3456640717 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f11e80ef940 con 0x7f11fc1062b0 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11e7e00640 1 -- 172.21.15.72:0/1282818487 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4004d20 con 0x5592c2971d60 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11e7e00640 1 -- 172.21.15.72:0/1282818487 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11f4004e80 con 0x5592c2971d60 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11e7e00640 1 -- 172.21.15.72:0/1282818487 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4005280 con 0x5592c2971d60 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.537+0000 7f11fb400640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 0x7f11e00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f11fb400640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 0x7f11e00a40e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x5592c29724c0 tx=0x7f11ec005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 -- 172.21.15.72:0/1282818487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 msgr2=0x5592c2970ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009f50 tx=0x7f11f40118d0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 -- 172.21.15.72:0/1282818487 shutdown_connections 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c297a570 0x5592c297c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c2970ff0 0x5592c2971470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 --2- 172.21.15.72:0/1282818487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2970ab0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 -- 172.21.15.72:0/1282818487 >> 172.21.15.72:0/1282818487 conn(0x5592c29384b0 msgr2=0x5592c296bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 -- 172.21.15.72:0/1282818487 shutdown_connections 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.538+0000 7f120500d740 1 -- 172.21.15.72:0/1282818487 wait complete. 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 -- start start 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 Processor -- start 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 0x5592c2977570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c297a570 0x5592c2977ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c28d75d0 con 0x5592c2971d60 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c287cb50 con 0x5592c2970ff0 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c287c7f0 con 0x5592c297a570 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.539+0000 7f120500d740 10 client.0 _ll_register_callbacks cb 0x5592c2890550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48454/0 (socket says 172.21.15.72:48454) 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 -- 172.21.15.72:0/1307614684 learned_addr learned my addr 172.21.15.72:0/1307614684 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11fb400640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 0x5592c2977570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11fbe00640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c297a570 0x5592c2977ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 -- 172.21.15.72:0/1307614684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c297a570 msgr2=0x5592c2977ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c297a570 0x5592c2977ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 -- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 msgr2=0x5592c2977570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 0x5592c2977570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 -- 172.21.15.72:0/1307614684 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f4009c00 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11fb400640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 0x5592c2977570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11faa00640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f11e80bbe90 tx=0x7f11e80098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344fa414dffce607 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.540+0000 7f11e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80ed100 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f11e8004440 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c28a9cd0 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e80bbcd0 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f120500d740 10 client.15411 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592c28d5c50 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592c28a9cd0 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.541+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11e804f060 con 0x5592c2971d60 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 10 client.15411 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 20 client.15411 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c298a3e0 0x5592c298c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 4 client.15411 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592c28cdce0 con 0x5592c298a3e0 2024-03-11T19:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f11fb400640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c298a3e0 0x5592c298c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f11fb400640 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c298a3e0 0x5592c298c800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e673fbbff7e673 server_cookie=65ce5b2c864fd109 in_seq=0 out_seq=0 2024-03-11T19:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.542+0000 7f11e4c00640 10 client.15411 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f11e4c00640 1 -- 172.21.15.72:0/1307614684 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592c28cdce0 con 0x5592c298a3e0 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f11e4c00640 10 client.15411 handle_command_reply: tid=0 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 client.15411 shutdown 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 msgr2=0x5592c2977ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f11e80bbe90 tx=0x7f11e80098f0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 shutdown_connections 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c298a3e0 0x5592c298c800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c297a570 0x5592c2977ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.543+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2971d60 0x5592c2977ab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f120500d740 1 --2- 172.21.15.72:0/1307614684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2970ff0 0x5592c2977570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 >> 172.21.15.72:0/1307614684 conn(0x5592c29384b0 msgr2=0x5592c296da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f11e4c00640 0 client.15411 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 shutdown_connections 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f120500d740 1 -- 172.21.15.72:0/1307614684 wait complete. 2024-03-11T19:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.544+0000 7f120500d740 20 client.15411 trim_cache size 0 max 16384 2024-03-11T19:23:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.559+0000 7f120500d740 1 Processor -- start 2024-03-11T19:23:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.559+0000 7f120500d740 1 -- start start 2024-03-11T19:23:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2977b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2975e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c261fe60 con 0x5592c2970ff0 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c296cae0 con 0x5592c2971d60 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f120500d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c297d1d0 con 0x5592c29384b0 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2975e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48456/0 (socket says 172.21.15.72:48456) 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f11fb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2977b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.560+0000 7f11faa00640 1 -- 172.21.15.72:0/1002003546 learned_addr learned my addr 172.21.15.72:0/1002003546 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 -- 172.21.15.72:0/1002003546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 msgr2=0x5592c2977b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2977b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 -- 172.21.15.72:0/1002003546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 msgr2=0x5592c2975e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2975e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 -- 172.21.15.72:0/1002003546 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c2872530 con 0x5592c2970ff0 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11fb400640 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11fbe00640 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2975e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11faa00640 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11e80bc300 tx=0x7f11e8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeb26342688b6a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11e7400640 1 -- 172.21.15.72:0/1002003546 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80c46a0 con 0x5592c2970ff0 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11e7400640 1 -- 172.21.15.72:0/1002003546 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e8004df0 con 0x5592c2970ff0 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.561+0000 7f11e7400640 1 -- 172.21.15.72:0/1002003546 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80ed0a0 con 0x5592c2970ff0 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 -- 172.21.15.72:0/1002003546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 msgr2=0x5592c2978040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11e80bc300 tx=0x7f11e8009880 comp rx=0 tx=0).stop 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 -- 172.21.15.72:0/1002003546 shutdown_connections 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c2975e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c2978040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 --2- 172.21.15.72:0/1002003546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 -- 172.21.15.72:0/1002003546 >> 172.21.15.72:0/1002003546 conn(0x5592c2967170 msgr2=0x5592c296b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 -- 172.21.15.72:0/1002003546 shutdown_connections 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.562+0000 7f120500d740 1 -- 172.21.15.72:0/1002003546 wait complete. 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.563+0000 7f120500d740 1 -- start start 2024-03-11T19:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.563+0000 7f120500d740 1 Processor -- start 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2981440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c297fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c2981cf0 con 0x5592c2970ff0 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c2980430 con 0x5592c2971d60 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c29805f0 con 0x5592c29384b0 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f120500d740 10 client.0 _ll_register_callbacks cb 0x7f11ec012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48472/0 (socket says 172.21.15.72:48472) 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 -- 172.21.15.72:0/940316932 learned_addr learned my addr 172.21.15.72:0/940316932 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11fb400640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2981440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11fbe00640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c297fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 -- 172.21.15.72:0/940316932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 msgr2=0x5592c2981440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2981440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 -- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 msgr2=0x5592c297fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c297fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11faa00640 1 -- 172.21.15.72:0/940316932 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec00f870 con 0x5592c2970ff0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.564+0000 7f11fbe00640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c297fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11faa00640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005820 tx=0x7f11e8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c4a798ae9d2923 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80c0330 con 0x5592c2970ff0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f11e8004440 con 0x5592c2970ff0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c28fcd30 con 0x5592c2970ff0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e8003e50 con 0x5592c2970ff0 2024-03-11T19:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f120500d740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592c2858b00 con 0x5592c2970ff0 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.565+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11e80a3020 con 0x5592c2970ff0 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11e803c070 con 0x5592c2970ff0 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 20 client.15417 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c297a570 0x5592c29e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 4 client.15417 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592c25d7600 con 0x5592c297a570 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.566+0000 7f11fb400640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c297a570 0x5592c29e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.567+0000 7f11e4c00640 10 client.15417 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.567+0000 7f11fb400640 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c297a570 0x5592c29e3790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876f7806b0a65bd8 server_cookie=4cb4b9b8a7486e1c in_seq=0 out_seq=0 2024-03-11T19:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.573+0000 7f11e4c00640 1 -- 172.21.15.72:0/940316932 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592c25d7600 con 0x5592c297a570 2024-03-11T19:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.573+0000 7f11e4c00640 10 client.15417 handle_command_reply: tid=0 2024-03-11T19:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.573+0000 7f120500d740 1 client.15417 shutdown 2024-03-11T19:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 msgr2=0x5592c297f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005820 tx=0x7f11e8005b50 comp rx=0 tx=0).stop 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 shutdown_connections 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5592c297a570 0x5592c29e3790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5592c2971d60 0x5592c297fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f11e4c00640 0 client.15417 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c2970ff0 0x5592c297f9b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 --2- 172.21.15.72:0/940316932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5592c29384b0 0x5592c2981440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 >> 172.21.15.72:0/940316932 conn(0x5592c2967170 msgr2=0x5592c2975e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 shutdown_connections 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 1 -- 172.21.15.72:0/940316932 wait complete. 2024-03-11T19:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.574+0000 7f120500d740 20 client.15417 trim_cache size 0 max 16384 2024-03-11T19:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 msgr2=0x7f11e00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 0x7f11e00a40e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x5592c29724c0 tx=0x7f11ec005f90 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 msgr2=0x7f11fc19fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc19fae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009230 tx=0x7f11e802c5c0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 shutdown_connections 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f11e00a1c20 0x7f11e00a40e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11fc199660 0x7f11fc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11fc1062b0 0x7f11fc19fae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 --2- 172.21.15.72:0/3456640717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f11fc1011e0 0x7f11fc19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 >> 172.21.15.72:0/3456640717 conn(0x7f11fc0fcac0 msgr2=0x7f11fc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 shutdown_connections 2024-03-11T19:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:38.575+0000 7f1200c00640 1 -- 172.21.15.72:0/3456640717 wait complete. 2024-03-11T19:23:38.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:38 smithi106 ceph-mon[35889]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:38.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:38 smithi072 ceph-mon[31695]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:38.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:38 smithi143 ceph-mon[36013]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:23:40.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:40 smithi106 ceph-mon[35889]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:40.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:40 smithi072 ceph-mon[31695]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:40.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:40 smithi143 ceph-mon[36013]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:42.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:42 smithi106 ceph-mon[35889]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-11T19:23:42.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:42 smithi072 ceph-mon[31695]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-11T19:23:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:42 smithi143 ceph-mon[36013]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-11T19:23:44.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:44 smithi106 ceph-mon[35889]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:44 smithi072 ceph-mon[31695]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:44 smithi143 ceph-mon[36013]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:46.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:46 smithi106 ceph-mon[35889]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:46.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:46 smithi072 ceph-mon[31695]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:46.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:46 smithi143 ceph-mon[36013]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:48 smithi106 ceph-mon[35889]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-11T19:23:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:48 smithi072 ceph-mon[31695]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-11T19:23:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:48 smithi143 ceph-mon[36013]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-11T19:23:50.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:50 smithi106 ceph-mon[35889]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:50.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:50 smithi072 ceph-mon[31695]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:50.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:50 smithi143 ceph-mon[36013]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:23:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 -- 172.21.15.72:0/1220874305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 msgr2=0x7f21c01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 --2- 172.21.15.72:0/1220874305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c01004b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009930 tx=0x7f21b002bc20 comp rx=0 tx=0).stop 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 -- 172.21.15.72:0/1220874305 shutdown_connections 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 --2- 172.21.15.72:0/1220874305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 0x7f21c010cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 --2- 172.21.15.72:0/1220874305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0100a80 0x7f21c0104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 --2- 172.21.15.72:0/1220874305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c01004b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 -- 172.21.15.72:0/1220874305 >> 172.21.15.72:0/1220874305 conn(0x7f21c00fb990 msgr2=0x7f21c00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 -- 172.21.15.72:0/1220874305 shutdown_connections 2024-03-11T19:23:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.707+0000 7f21bfe00640 1 -- 172.21.15.72:0/1220874305 wait complete. 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.708+0000 7f21bfe00640 1 Processor -- start 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.708+0000 7f21bfe00640 1 -- start start 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.708+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 0x7f21c019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c01996b0 0x7f21c0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bfe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c019a180 con 0x7f21c01000d0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bfe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c019a2f0 con 0x7f21c01051a0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bfe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c019a4b0 con 0x7f21c01996b0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48094/0 (socket says 172.21.15.72:48094) 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 -- 172.21.15.72:0/2218584787 learned_addr learned my addr 172.21.15.72:0/2218584787 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21be000640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 0x7f21c019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bf400640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c01996b0 0x7f21c0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 -- 172.21.15.72:0/2218584787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c01996b0 msgr2=0x7f21c0199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c01996b0 0x7f21c0199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 -- 172.21.15.72:0/2218584787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 msgr2=0x7f21c019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 0x7f21c019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.709+0000 7f21bea00640 1 -- 172.21.15.72:0/2218584787 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0009560 con 0x7f21c01000d0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21bea00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009230 tx=0x7f21b002c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21b7400640 1 -- 172.21.15.72:0/2218584787 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0004350 con 0x7f21c01000d0 2024-03-11T19:23:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21b7400640 1 -- 172.21.15.72:0/2218584787 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21b00044b0 con 0x7f21c01000d0 2024-03-11T19:23:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c01a63f0 con 0x7f21c01000d0 2024-03-11T19:23:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21b7400640 1 -- 172.21.15.72:0/2218584787 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0003d70 con 0x7f21c01000d0 2024-03-11T19:23:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.710+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c01a68d0 con 0x7f21c01000d0 2024-03-11T19:23:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.713+0000 7f21b7400640 1 -- 172.21.15.72:0/2218584787 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) v1 ==== 136455+0+0 (secure 0 0 0) 0x7f21b0004df0 con 0x7f21c01000d0 2024-03-11T19:23:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 Processor -- start 2024-03-11T19:23:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 -- start start 2024-03-11T19:23:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21b7400640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 0x7f21a40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21b7400640 1 -- 172.21.15.72:0/2218584787 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f21b00efd00 con 0x7f21c01000d0 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21bf400640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 0x7f21a40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a7e570 0x558060a80960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060a30620 con 0x558060a74ff0 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580609e28c0 con 0x558060a7e570 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.715+0000 7f21c898d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580609dfdf0 con 0x558060a75d60 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48108/0 (socket says 172.21.15.72:48108) 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 -- 172.21.15.72:0/877231033 learned_addr learned my addr 172.21.15.72:0/877231033 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21bea00640 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21bf400640 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a7e570 0x558060a80960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 -- 172.21.15.72:0/877231033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 msgr2=0x558060a74ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a74ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 -- 172.21.15.72:0/877231033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a7e570 msgr2=0x558060a80960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a7e570 0x558060a80960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21bf400640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 0x7f21a40a40e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f21b8005fd0 tx=0x7f21b8010740 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.716+0000 7f21be000640 1 -- 172.21.15.72:0/877231033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806095cb00 con 0x558060a74ff0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21be000640 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009c60 tx=0x7f21ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69fa4251b2ad1c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21abe00640 1 -- 172.21.15.72:0/877231033 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004230 con 0x558060a74ff0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21abe00640 1 -- 172.21.15.72:0/877231033 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21ac004390 con 0x558060a74ff0 2024-03-11T19:23:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21abe00640 1 -- 172.21.15.72:0/877231033 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac019780 con 0x558060a74ff0 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21c898d740 1 -- 172.21.15.72:0/877231033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 msgr2=0x558060a75470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.717+0000 7f21c898d740 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009c60 tx=0x7f21ac011810 comp rx=0 tx=0).stop 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 -- 172.21.15.72:0/877231033 shutdown_connections 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a7e570 0x558060a80960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a74ff0 0x558060a75470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 --2- 172.21.15.72:0/877231033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 -- 172.21.15.72:0/877231033 >> 172.21.15.72:0/877231033 conn(0x558060a3c4b0 msgr2=0x558060a6fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 -- 172.21.15.72:0/877231033 shutdown_connections 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 -- 172.21.15.72:0/877231033 wait complete. 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 -- start start 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.718+0000 7f21c898d740 1 Processor -- start 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 0x558060a7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580609db5d0 con 0x558060a7e570 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060980b50 con 0x558060a75d60 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580609807f0 con 0x558060a74ff0 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21c898d740 10 client.0 _ll_register_callbacks cb 0x558060994550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21be000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21bf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21be000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:50604/0 (socket says 172.21.15.72:50604) 2024-03-11T19:23:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21be000640 1 -- 172.21.15.72:0/38200049 learned_addr learned my addr 172.21.15.72:0/38200049 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21bea00640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 0x558060a7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.719+0000 7f21bea00640 1 -- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 msgr2=0x558060a7baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bea00640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bea00640 1 -- 172.21.15.72:0/38200049 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 msgr2=0x558060a7c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bea00640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a7c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bea00640 1 -- 172.21.15.72:0/38200049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac009890 con 0x558060a74ff0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21be000640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bf400640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a7c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21bea00640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 0x558060a7b5b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21b00ed0c0 tx=0x7f21b0003840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb4b6a05bb1398e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b00bc4f0 con 0x558060a74ff0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f21b00c0640 con 0x558060a74ff0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580609adcd0 con 0x558060a74ff0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21b00c4c90 con 0x558060a74ff0 2024-03-11T19:23:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21c898d740 10 client.25522 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.720+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5580609d9c50 con 0x558060a74ff0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5580609adcd0 con 0x558060a74ff0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f21b003c070 con 0x558060a74ff0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 10 client.25522 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 20 client.25522 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060a8e4f0 0x558060a908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 4 client.25522 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.721+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580609d1ce0 con 0x558060a8e4f0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.722+0000 7f21be000640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060a8e4f0 0x558060a908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.722+0000 7f21be000640 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060a8e4f0 0x558060a908e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26326b455777fb6c server_cookie=a041da5405c4c9db in_seq=0 out_seq=0 2024-03-11T19:23:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.722+0000 7f21a8c00640 10 client.25522 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21a8c00640 1 -- 172.21.15.72:0/38200049 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580609d1ce0 con 0x558060a8e4f0 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21a8c00640 10 client.25522 handle_command_reply: tid=0 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 client.25522 shutdown 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 msgr2=0x558060a7b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 0x558060a7b5b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21b00ed0c0 tx=0x7f21b0003840 comp rx=0 tx=0).stop 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 shutdown_connections 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060a8e4f0 0x558060a908e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a7c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a75d60 0x558060a7baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21a8c00640 0 client.25522 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 --2- 172.21.15.72:0/38200049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a74ff0 0x558060a7b5b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 >> 172.21.15.72:0/38200049 conn(0x558060a3c4b0 msgr2=0x558060a71a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 shutdown_connections 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 1 -- 172.21.15.72:0/38200049 wait complete. 2024-03-11T19:23:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.723+0000 7f21c898d740 20 client.25522 trim_cache size 0 max 16384 2024-03-11T19:23:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.738+0000 7f21c898d740 1 Processor -- start 2024-03-11T19:23:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.738+0000 7f21c898d740 1 -- start start 2024-03-11T19:23:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.738+0000 7f21c898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.738+0000 7f21c898d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21c898d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21c898d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060723e60 con 0x558060a7e570 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21c898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580609adcd0 con 0x558060a74ff0 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21c898d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060a7c580 con 0x558060a75d60 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21bf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21bea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:46158/0 (socket says 172.21.15.72:46158) 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 -- 172.21.15.72:0/3863063512 learned_addr learned my addr 172.21.15.72:0/3863063512 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 -- 172.21.15.72:0/3863063512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 msgr2=0x558060a6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 -- 172.21.15.72:0/3863063512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 msgr2=0x558060a79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.739+0000 7f21be000640 1 -- 172.21.15.72:0/3863063512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558060976530 con 0x558060a75d60 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.740+0000 7f21be000640 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009c60 tx=0x7f21ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff581eaee73043a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.740+0000 7f21ab400640 1 -- 172.21.15.72:0/3863063512 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac005410 con 0x558060a75d60 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.740+0000 7f21ab400640 1 -- 172.21.15.72:0/3863063512 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21ac02e070 con 0x558060a75d60 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.740+0000 7f21ab400640 1 -- 172.21.15.72:0/3863063512 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac003850 con 0x558060a75d60 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- 172.21.15.72:0/3863063512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 msgr2=0x558060a6fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009c60 tx=0x7f21ac004270 comp rx=0 tx=0).stop 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- 172.21.15.72:0/3863063512 shutdown_connections 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a7e570 0x558060a79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a75d60 0x558060a6fd50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 --2- 172.21.15.72:0/3863063512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- 172.21.15.72:0/3863063512 >> 172.21.15.72:0/3863063512 conn(0x558060a3c4b0 msgr2=0x558060a7ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- 172.21.15.72:0/3863063512 shutdown_connections 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- 172.21.15.72:0/3863063512 wait complete. 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.741+0000 7f21c898d740 1 -- start start 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 Processor -- start 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a7e570 0x558060ae39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060a84120 con 0x558060a75d60 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060ae3f00 con 0x558060a74ff0 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060ae4090 con 0x558060a7e570 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21c898d740 10 client.0 _ll_register_callbacks cb 0x7f21b8010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48138/0 (socket says 172.21.15.72:48138) 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21be000640 1 -- 172.21.15.72:0/2862548596 learned_addr learned my addr 172.21.15.72:0/2862548596 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.742+0000 7f21bea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21bf400640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a7e570 0x558060ae39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 -- 172.21.15.72:0/2862548596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a7e570 msgr2=0x558060ae39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a7e570 0x558060ae39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 -- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 msgr2=0x558060a834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 -- 172.21.15.72:0/2862548596 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806095cb00 con 0x558060a75d60 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21bea00640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21be000640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011d10 tx=0x7f21ac004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3521a54d1350e90e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac039070 con 0x558060a75d60 2024-03-11T19:23:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f21ac0053d0 con 0x558060a75d60 2024-03-11T19:23:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.743+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558060a00d30 con 0x558060a75d60 2024-03-11T19:23:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ac005590 con 0x558060a75d60 2024-03-11T19:23:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 10 client.15435 fetch_fsmap learned FSMap version 25 2024-03-11T19:23:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55806099c2c0 con 0x558060a75d60 2024-03-11T19:23:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558060a00d30 con 0x558060a75d60 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f21ac03c030 con 0x558060a75d60 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 20 client.15435 populate_metadata read hostname 'smithi072' 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060ae54d0 0x558060ae78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 4 client.15435 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.744+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580606db600 con 0x558060ae54d0 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.745+0000 7f21bea00640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060ae54d0 0x558060ae78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.745+0000 7f21bea00640 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060ae54d0 0x558060ae78f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af09f030aecb0041 server_cookie=a4848ee46372afd9 in_seq=0 out_seq=0 2024-03-11T19:23:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.745+0000 7f21a8c00640 10 client.15435 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.752+0000 7f21a8c00640 1 -- 172.21.15.72:0/2862548596 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580606db600 con 0x558060ae54d0 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.752+0000 7f21a8c00640 10 client.15435 handle_command_reply: tid=0 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.752+0000 7f21c898d740 1 client.15435 shutdown 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.752+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 msgr2=0x558060a839e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.752+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011d10 tx=0x7f21ac004520 comp rx=0 tx=0).stop 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 shutdown_connections 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558060ae54d0 0x558060ae78f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558060a7e570 0x558060ae39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558060a75d60 0x558060a839e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 --2- 172.21.15.72:0/2862548596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558060a74ff0 0x558060a834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21a8c00640 0 client.15435 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 >> 172.21.15.72:0/2862548596 conn(0x558060a3c4b0 msgr2=0x558060a79e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 shutdown_connections 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 1 -- 172.21.15.72:0/2862548596 wait complete. 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.753+0000 7f21c898d740 20 client.15435 trim_cache size 0 max 16384 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 msgr2=0x7f21a40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 0x7f21a40a40e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f21b8005fd0 tx=0x7f21b8010740 comp rx=0 tx=0).stop 2024-03-11T19:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 msgr2=0x7f21c019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009230 tx=0x7f21b002c6b0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 shutdown_connections 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f21a40a1c20 0x7f21a40a40e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c01996b0 0x7f21c0199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c01051a0 0x7f21c019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 --2- 172.21.15.72:0/2218584787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21c01000d0 0x7f21c019f5f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 >> 172.21.15.72:0/2218584787 conn(0x7f21c00fb990 msgr2=0x7f21c00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 shutdown_connections 2024-03-11T19:23:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:23:51.754+0000 7f21bfe00640 1 -- 172.21.15.72:0/2218584787 wait complete. 2024-03-11T19:23:52.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:52 smithi106 ceph-mon[35889]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-11T19:23:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:52 smithi072 ceph-mon[31695]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-11T19:23:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:52 smithi143 ceph-mon[36013]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-03-11T19:23:54.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:54 smithi106 ceph-mon[35889]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:54.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:54 smithi072 ceph-mon[31695]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:54.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:54 smithi143 ceph-mon[36013]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:56.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:56 smithi072 ceph-mon[31695]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:56.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:56 smithi143 ceph-mon[36013]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:56 smithi106 ceph-mon[35889]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-11T19:23:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:23:58 smithi072 ceph-mon[31695]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:23:58 smithi143 ceph-mon[36013]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:23:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:23:58 smithi106 ceph-mon[35889]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:24:00.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:00 smithi072 ceph-mon[31695]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:24:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:00 smithi143 ceph-mon[36013]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:24:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:00 smithi106 ceph-mon[35889]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:24:01.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:24:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:24:00.998+0000 7fd2c3a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:24:01.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:24:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:24:00.998+0000 7fd2c3a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:24:01.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:24:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:24:00.998+0000 7fd2c3a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:24:01.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:24:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:24:00.998+0000 7fd2c3a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:24:01.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:24:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:24:00.998+0000 7fd2c3a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:24:02.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:02 smithi072 ceph-mon[31695]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:24:02.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:02 smithi072 ceph-mon[31695]: mgrmap e27: x(active, since 52s), standbys: y, z 2024-03-11T19:24:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:02 smithi143 ceph-mon[36013]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:24:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:02 smithi143 ceph-mon[36013]: mgrmap e27: x(active, since 52s), standbys: y, z 2024-03-11T19:24:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:02 smithi106 ceph-mon[35889]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:24:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:02 smithi106 ceph-mon[35889]: mgrmap e27: x(active, since 52s), standbys: y, z 2024-03-11T19:24:04.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:04 smithi072 ceph-mon[31695]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:04.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:04 smithi143 ceph-mon[36013]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:04 smithi106 ceph-mon[35889]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 -- 172.21.15.72:0/2176724739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe150101ba0 msgr2=0x7fe150105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 --2- 172.21.15.72:0/2176724739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe150101ba0 0x7fe150105be0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009860 tx=0x7fe13c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 -- 172.21.15.72:0/2176724739 shutdown_connections 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 --2- 172.21.15.72:0/2176724739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501062c0 0x7fe15010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 --2- 172.21.15.72:0/2176724739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe150101ba0 0x7fe150105be0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 --2- 172.21.15.72:0/2176724739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe1501011f0 0x7fe1501015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 -- 172.21.15.72:0/2176724739 >> 172.21.15.72:0/2176724739 conn(0x7fe1500fca90 msgr2=0x7fe1500feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 -- 172.21.15.72:0/2176724739 shutdown_connections 2024-03-11T19:24:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.897+0000 7fe155200640 1 -- 172.21.15.72:0/2176724739 wait complete. 2024-03-11T19:24:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.898+0000 7fe155200640 1 Processor -- start 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.898+0000 7fe155200640 1 -- start start 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 0x7fe15019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe150199660 0x7fe150199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a130 con 0x7fe150199660 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a2a0 con 0x7fe1501011f0 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe155200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a460 con 0x7fe1501062c0 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe14f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe14f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45736/0 (socket says 172.21.15.72:45736) 2024-03-11T19:24:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe14f400640 1 -- 172.21.15.72:0/3119772673 learned_addr learned my addr 172.21.15.72:0/3119772673 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.899+0000 7fe14ea00640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 0x7fe15019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14fe00640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe150199660 0x7fe150199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 -- 172.21.15.72:0/3119772673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 msgr2=0x7fe15019fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 0x7fe15019fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 -- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe150199660 msgr2=0x7fe150199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe150199660 0x7fe150199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 -- 172.21.15.72:0/3119772673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148009840 con 0x7fe1501011f0 2024-03-11T19:24:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14ea00640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 0x7fe15019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe14f400640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe140009e70 tx=0x7fe14000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe147e00640 1 -- 172.21.15.72:0/3119772673 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000c8b0 con 0x7fe1501011f0 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c009490 con 0x7fe1501011f0 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe147e00640 1 -- 172.21.15.72:0/3119772673 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe140005020 con 0x7fe1501011f0 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.900+0000 7fe147e00640 1 -- 172.21.15.72:0/3119772673 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140004b20 con 0x7fe1501011f0 2024-03-11T19:24:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.901+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe150108140 con 0x7fe1501011f0 2024-03-11T19:24:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.904+0000 7fe1593dd740 1 Processor -- start 2024-03-11T19:24:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 -- start start 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe147e00640 1 -- 172.21.15.72:0/3119772673 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 27) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fe1400054e0 con 0x7fe1501011f0 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a30ff0 0x564321a31470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a4d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643219ec620 con 0x564321a4b580 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432199e8c0 con 0x564321a31d60 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.905+0000 7fe1593dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432199bdf0 con 0x564321a30ff0 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45738/0 (socket says 172.21.15.72:45738) 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 -- 172.21.15.72:0/168247385 learned_addr learned my addr 172.21.15.72:0/168247385 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14ea00640 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a4d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 -- 172.21.15.72:0/168247385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a30ff0 msgr2=0x564321a31470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:24:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a30ff0 0x564321a31470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 -- 172.21.15.72:0/168247385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 msgr2=0x564321a4d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a4d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.906+0000 7fe14fe00640 1 -- 172.21.15.72:0/168247385 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564321918b00 con 0x564321a31d60 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe14fe00640 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe148009fa0 tx=0x7fe14800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606b8fa8e99c6b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe147e00640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 0x7fe1340a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe147e00640 1 -- 172.21.15.72:0/3119772673 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fe1400cdd00 con 0x7fe1501011f0 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe144c00640 1 -- 172.21.15.72:0/168247385 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148004d20 con 0x564321a31d60 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe144c00640 1 -- 172.21.15.72:0/168247385 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe148004e80 con 0x564321a31d60 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.907+0000 7fe144c00640 1 -- 172.21.15.72:0/168247385 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148005680 con 0x564321a31d60 2024-03-11T19:24:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe14f400640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 0x7fe1340a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 -- 172.21.15.72:0/168247385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 msgr2=0x564321a30ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe148009fa0 tx=0x7fe14800f7f0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 -- 172.21.15.72:0/168247385 shutdown_connections 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a4d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a30ff0 0x564321a31470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 --2- 172.21.15.72:0/168247385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a31d60 0x564321a30ab0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 -- 172.21.15.72:0/168247385 >> 172.21.15.72:0/168247385 conn(0x5643219f84b0 msgr2=0x564321a2bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 -- 172.21.15.72:0/168247385 shutdown_connections 2024-03-11T19:24:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe1593dd740 1 -- 172.21.15.72:0/168247385 wait complete. 2024-03-11T19:24:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.908+0000 7fe14f400640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 0x7fe1340a41f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1400cb060 tx=0x7fe1400094c0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 -- start start 2024-03-11T19:24:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 Processor -- start 2024-03-11T19:24:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a3c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a37650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643219975d0 con 0x564321a31d60 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432193cb50 con 0x564321a30ff0 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432193c7f0 con 0x564321a4b580 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.909+0000 7fe1593dd740 10 client.0 _ll_register_callbacks cb 0x564321950550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45748/0 (socket says 172.21.15.72:45748) 2024-03-11T19:24:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 -- 172.21.15.72:0/669302395 learned_addr learned my addr 172.21.15.72:0/669302395 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14ea00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a3c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 -- 172.21.15.72:0/669302395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 msgr2=0x564321a37650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a37650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 -- 172.21.15.72:0/669302395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 msgr2=0x564321a3c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a3c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 -- 172.21.15.72:0/669302395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe148009c80 con 0x564321a30ff0 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14ea00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a3c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe14fe00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe148009fd0 tx=0x7fe148005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d707f3d93b872e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.910+0000 7fe139600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148011500 con 0x564321a30ff0 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564321969cd0 con 0x564321a30ff0 2024-03-11T19:24:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fe148011660 con 0x564321a30ff0 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe148011820 con 0x564321a30ff0 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe1593dd740 10 client.25550 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564321995c50 con 0x564321a30ff0 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.911+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564321969cd0 con 0x564321a30ff0 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe14801b9e0 con 0x564321a30ff0 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 10 client.25550 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 20 client.25550 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321a47730 0x564321a49b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 4 client.25550 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56432198dce0 con 0x564321a47730 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.912+0000 7fe14ea00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321a47730 0x564321a49b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.913+0000 7fe14ea00640 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321a47730 0x564321a49b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d6eb204075662d server_cookie=83e335d9e43e3032 in_seq=0 out_seq=0 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.913+0000 7fe139600640 10 client.25550 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.913+0000 7fe139600640 1 -- 172.21.15.72:0/669302395 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56432198dce0 con 0x564321a47730 2024-03-11T19:24:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.913+0000 7fe139600640 10 client.25550 handle_command_reply: tid=0 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.913+0000 7fe1593dd740 1 client.25550 shutdown 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 msgr2=0x564321a3c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe148009fd0 tx=0x7fe148005300 comp rx=0 tx=0).stop 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 shutdown_connections 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321a47730 0x564321a49b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a3c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 --2- 172.21.15.72:0/669302395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a3c250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 >> 172.21.15.72:0/669302395 conn(0x5643219f84b0 msgr2=0x564321a2da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe139600640 0 client.25550 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 shutdown_connections 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 1 -- 172.21.15.72:0/669302395 wait complete. 2024-03-11T19:24:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.914+0000 7fe1593dd740 20 client.25550 trim_cache size 0 max 16384 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 Processor -- start 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 -- start start 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a36ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643216dfe60 con 0x564321a31d60 2024-03-11T19:24:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x564321a4ddd0 con 0x564321a30ff0 2024-03-11T19:24:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe1593dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x564321a32310 con 0x564321a4b580 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe14ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe14ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44344/0 (socket says 172.21.15.72:44344) 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.928+0000 7fe14ea00640 1 -- 172.21.15.72:0/4105911399 learned_addr learned my addr 172.21.15.72:0/4105911399 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14fe00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a36ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14fe00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 -- 172.21.15.72:0/4105911399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 msgr2=0x564321a348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 -- 172.21.15.72:0/4105911399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 msgr2=0x564321a36ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a36ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 -- 172.21.15.72:0/4105911399 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564321932530 con 0x564321a31d60 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14fe00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.929+0000 7fe14ea00640 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009250 tx=0x7fe13c02c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca631acf3779ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe13be00640 1 -- 172.21.15.72:0/4105911399 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c043ea0 con 0x564321a31d60 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe13be00640 1 -- 172.21.15.72:0/4105911399 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe13c043070 con 0x564321a31d60 2024-03-11T19:24:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe13be00640 1 -- 172.21.15.72:0/4105911399 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c005540 con 0x564321a31d60 2024-03-11T19:24:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 -- 172.21.15.72:0/4105911399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 msgr2=0x564321a37420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009250 tx=0x7fe13c02c1f0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 -- 172.21.15.72:0/4105911399 shutdown_connections 2024-03-11T19:24:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a4b580 0x564321a348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a31d60 0x564321a37420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 --2- 172.21.15.72:0/4105911399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a36ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.930+0000 7fe1593dd740 1 -- 172.21.15.72:0/4105911399 >> 172.21.15.72:0/4105911399 conn(0x5643219f84b0 msgr2=0x564321a35f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.931+0000 7fe1593dd740 1 -- 172.21.15.72:0/4105911399 shutdown_connections 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.931+0000 7fe1593dd740 1 -- 172.21.15.72:0/4105911399 wait complete. 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.931+0000 7fe1593dd740 1 -- start start 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.931+0000 7fe1593dd740 1 Processor -- start 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a31d60 0x564321a51540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a9faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564321a51ce0 con 0x564321a4b580 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x564321a51e50 con 0x564321a30ff0 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x564321a9ffe0 con 0x564321a31d60 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe1593dd740 10 client.0 _ll_register_callbacks cb 0x7fe148015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:45758/0 (socket says 172.21.15.72:45758) 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 -- 172.21.15.72:0/1123635056 learned_addr learned my addr 172.21.15.72:0/1123635056 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14ea00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a31d60 0x564321a51540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a9faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 -- 172.21.15.72:0/1123635056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a31d60 msgr2=0x564321a51540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.932+0000 7fe14fe00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a31d60 0x564321a51540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe14fe00640 1 -- 172.21.15.72:0/1123635056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 msgr2=0x564321a9faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe14fe00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a9faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe14fe00640 1 -- 172.21.15.72:0/1123635056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15010e6a0 con 0x564321a30ff0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe14fe00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1480052d0 tx=0x7fe148005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7264324c56b56d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe139600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148004a30 con 0x564321a30ff0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fe148005810 con 0x564321a30ff0 2024-03-11T19:24:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.933+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643219bcd30 con 0x564321a30ff0 2024-03-11T19:24:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1480059d0 con 0x564321a30ff0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 10 client.25556 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5643219582c0 con 0x564321a30ff0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5643219bcd30 con 0x564321a30ff0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe14802a8d0 con 0x564321a30ff0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 10 client.25556 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 20 client.25556 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321aa13a0 0x564321aa3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 4 client.25556 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.934+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564321697600 con 0x564321aa13a0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.935+0000 7fe14ea00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321aa13a0 0x564321aa3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.936+0000 7fe14ea00640 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321aa13a0 0x564321aa3790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953f6a8d17dd5864 server_cookie=386bcda9f9682e50 in_seq=0 out_seq=0 2024-03-11T19:24:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.936+0000 7fe139600640 10 client.25556 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe139600640 1 -- 172.21.15.72:0/1123635056 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564321697600 con 0x564321aa13a0 2024-03-11T19:24:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe139600640 10 client.25556 handle_command_reply: tid=0 2024-03-11T19:24:04.945 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 client.25556 shutdown 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 msgr2=0x564321a51000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1480052d0 tx=0x7fe148005260 comp rx=0 tx=0).stop 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 shutdown_connections 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x564321aa13a0 0x564321aa3790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564321a4b580 0x564321a9faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564321a31d60 0x564321a51540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 --2- 172.21.15.72:0/1123635056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564321a30ff0 0x564321a51000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe139600640 0 client.25556 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 >> 172.21.15.72:0/1123635056 conn(0x5643219f84b0 msgr2=0x564321a36e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 shutdown_connections 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 1 -- 172.21.15.72:0/1123635056 wait complete. 2024-03-11T19:24:04.946 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.942+0000 7fe1593dd740 20 client.25556 trim_cache size 0 max 16384 2024-03-11T19:24:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:24:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:24:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:24:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 msgr2=0x7fe1340a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 0x7fe1340a41f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe1400cb060 tx=0x7fe1400094c0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 msgr2=0x7fe15019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe140009e70 tx=0x7fe14000ed90 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 shutdown_connections 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe1340a1d30 0x7fe1340a41f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe150199660 0x7fe150199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1501062c0 0x7fe15019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 --2- 172.21.15.72:0/3119772673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1501011f0 0x7fe15019f5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.943+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 >> 172.21.15.72:0/3119772673 conn(0x7fe1500fca90 msgr2=0x7fe1500fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.944+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 shutdown_connections 2024-03-11T19:24:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:04.944+0000 7fe155200640 1 -- 172.21.15.72:0/3119772673 wait complete. 2024-03-11T19:24:06.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:06 smithi072 ceph-mon[31695]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:06 smithi143 ceph-mon[36013]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:06 smithi106 ceph-mon[35889]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:24:08.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:08 smithi072 ceph-mon[31695]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-11T19:24:08.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:08 smithi143 ceph-mon[36013]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-11T19:24:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:08 smithi106 ceph-mon[35889]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-11T19:24:10.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:10 smithi072 ceph-mon[31695]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:10.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:10 smithi143 ceph-mon[36013]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:10 smithi106 ceph-mon[35889]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:12.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:12 smithi143 ceph-mon[36013]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:12 smithi106 ceph-mon[35889]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:12 smithi072 ceph-mon[31695]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:14.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:14 smithi143 ceph-mon[36013]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:14 smithi106 ceph-mon[35889]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:14 smithi072 ceph-mon[31695]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:16.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:16 smithi143 ceph-mon[36013]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:16 smithi106 ceph-mon[35889]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:16 smithi072 ceph-mon[31695]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:24:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 -- 172.21.15.72:0/2760875344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f8010d8f0 msgr2=0x7f6f8010fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/2760875344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f8010d8f0 0x7f6f8010fce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009be0 tx=0x7f6f7802bef0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 -- 172.21.15.72:0/2760875344 shutdown_connections 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/2760875344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f8010d8f0 0x7f6f8010fce0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/2760875344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f801006f0 0x7f6f8010d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/2760875344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f800ffd40 0x7f6f80100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 -- 172.21.15.72:0/2760875344 >> 172.21.15.72:0/2760875344 conn(0x7f6f800fb990 msgr2=0x7f6f800fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 -- 172.21.15.72:0/2760875344 shutdown_connections 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.069+0000 7f6f7fe00640 1 -- 172.21.15.72:0/2760875344 wait complete. 2024-03-11T19:24:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.071+0000 7f6f7fe00640 1 Processor -- start 2024-03-11T19:24:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.071+0000 7f6f7fe00640 1 -- start start 2024-03-11T19:24:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.071+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 0x7f6f80101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.071+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 0x7f6f801024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.071+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 0x7f6f80102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f801a9e10 con 0x7f6f8010d8f0 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80199520 con 0x7f6f801006f0 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f801996e0 con 0x7f6f800ffd40 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 0x7f6f801024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 0x7f6f801024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:60110/0 (socket says 172.21.15.72:60110) 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7e000640 1 -- 172.21.15.72:0/713463982 learned_addr learned my addr 172.21.15.72:0/713463982 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 0x7f6f80101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.072+0000 7f6f7f400640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 0x7f6f80102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 -- 172.21.15.72:0/713463982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 msgr2=0x7f6f801024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 0x7f6f801024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 -- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 msgr2=0x7f6f80102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 0x7f6f80102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 -- 172.21.15.72:0/713463982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78009840 con 0x7f6f800ffd40 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7f400640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 0x7f6f80102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.073+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 0x7f6f80101f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009fd0 tx=0x7f6f7000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.074+0000 7f6f84a00640 1 -- 172.21.15.72:0/713463982 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000ccc0 con 0x7f6f800ffd40 2024-03-11T19:24:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.074+0000 7f6f84a00640 1 -- 172.21.15.72:0/713463982 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f7000ee70 con 0x7f6f800ffd40 2024-03-11T19:24:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.074+0000 7f6f84a00640 1 -- 172.21.15.72:0/713463982 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70004a00 con 0x7f6f800ffd40 2024-03-11T19:24:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.074+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f801999c0 con 0x7f6f800ffd40 2024-03-11T19:24:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.075+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f80199e50 con 0x7f6f800ffd40 2024-03-11T19:24:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.078+0000 7f6f889f5740 1 Processor -- start 2024-03-11T19:24:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.078+0000 7f6f889f5740 1 -- start start 2024-03-11T19:24:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d46706ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46707470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d46712960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d466c2620 con 0x559d46707d60 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d466748c0 con 0x559d46706ff0 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.079+0000 7f6f889f5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46671df0 con 0x559d46710570 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.080+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d46706ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.080+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d46712960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.080+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d46712960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:55654/0 (socket says 172.21.15.72:55654) 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.080+0000 7f6f7e000640 1 -- 172.21.15.72:0/2211670753 learned_addr learned my addr 172.21.15.72:0/2211670753 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f84a00640 1 -- 172.21.15.72:0/713463982 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 136512+0+0 (secure 0 0 0) 0x7f6f70004b60 con 0x7f6f800ffd40 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 -- 172.21.15.72:0/2211670753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 msgr2=0x559d46712960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d46712960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 -- 172.21.15.72:0/2211670753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 msgr2=0x559d46707470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46707470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 -- 172.21.15.72:0/2211670753 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d465eeb00 con 0x559d46707d60 2024-03-11T19:24:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.081+0000 7f6f7f400640 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d46706ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f7802c500 tx=0x7f6f7802cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ec50522f6c97a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.082+0000 7f6f6be00640 1 -- 172.21.15.72:0/2211670753 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7802ca90 con 0x559d46707d60 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.082+0000 7f6f6be00640 1 -- 172.21.15.72:0/2211670753 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f78003970 con 0x559d46707d60 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.082+0000 7f6f6be00640 1 -- 172.21.15.72:0/2211670753 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780045d0 con 0x559d46707d60 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.083+0000 7f6f889f5740 1 -- 172.21.15.72:0/2211670753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 msgr2=0x559d46706ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.083+0000 7f6f889f5740 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d46706ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f7802c500 tx=0x7f6f7802cd60 comp rx=0 tx=0).stop 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.083+0000 7f6f889f5740 1 -- 172.21.15.72:0/2211670753 shutdown_connections 2024-03-11T19:24:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d46712960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46707470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 --2- 172.21.15.72:0/2211670753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d46706ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 -- 172.21.15.72:0/2211670753 >> 172.21.15.72:0/2211670753 conn(0x559d466ce4b0 msgr2=0x559d46701ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 -- 172.21.15.72:0/2211670753 shutdown_connections 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 -- 172.21.15.72:0/2211670753 wait complete. 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.084+0000 7f6f889f5740 1 -- start start 2024-03-11T19:24:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 Processor -- start 2024-03-11T19:24:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f84a00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 0x7f6f640a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4666d5d0 con 0x559d46707d60 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46612b50 con 0x559d46706ff0 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d466127f0 con 0x559d46710570 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.085+0000 7f6f889f5740 10 client.0 _ll_register_callbacks cb 0x559d46626550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f84a00640 1 -- 172.21.15.72:0/713463982 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f6f70014070 con 0x7f6f800ffd40 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:60130/0 (socket says 172.21.15.72:60130) 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1381005472 learned_addr learned my addr 172.21.15.72:0/1381005472 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.086+0000 7f6f7f400640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 0x7f6f640a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1381005472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 msgr2=0x559d4670e0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670e0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1381005472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 msgr2=0x559d4670db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1381005472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f801108e0 con 0x559d46706ff0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.087+0000 7f6f7e000640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f70096ee0 tx=0x7f6f7009a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cbea3b13d0b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f68c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7009ac10 con 0x559d46706ff0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f6f7009a510 con 0x559d46706ff0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d4663fcd0 con 0x559d46706ff0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f7009e660 con 0x559d46706ff0 2024-03-11T19:24:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f7f400640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 0x7f6f640a4120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f7802c430 tx=0x7f6f7802bef0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.088+0000 7f6f889f5740 10 client.25565 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.089+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d4666bc50 con 0x559d46706ff0 2024-03-11T19:24:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.089+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f7007d020 con 0x559d46706ff0 2024-03-11T19:24:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f700cb470 con 0x559d46706ff0 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 10 client.25565 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 20 client.25565 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46720480 0x559d46722870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 4 client.25565 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d46663ce0 con 0x559d46720480 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.090+0000 7f6f7e000640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46720480 0x559d46722870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.091+0000 7f6f7e000640 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46720480 0x559d46722870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c1b8d59795669a server_cookie=d16f0a7d5c7aa818 in_seq=0 out_seq=0 2024-03-11T19:24:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.091+0000 7f6f68c00640 10 client.25565 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f68c00640 1 -- 172.21.15.72:0/1381005472 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d46663ce0 con 0x559d46720480 2024-03-11T19:24:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f68c00640 10 client.25565 handle_command_reply: tid=0 2024-03-11T19:24:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f889f5740 1 client.25565 shutdown 2024-03-11T19:24:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 msgr2=0x559d4670d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f70096ee0 tx=0x7f6f7009a2e0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 shutdown_connections 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.092+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46720480 0x559d46722870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1381005472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670d640 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 >> 172.21.15.72:0/1381005472 conn(0x559d466ce4b0 msgr2=0x559d46703a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f68c00640 0 client.25565 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 shutdown_connections 2024-03-11T19:24:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 1 -- 172.21.15.72:0/1381005472 wait complete. 2024-03-11T19:24:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.093+0000 7f6f889f5740 20 client.25565 trim_cache size 0 max 16384 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.120+0000 7f6f889f5740 1 Processor -- start 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.120+0000 7f6f889f5740 1 -- start start 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d463b5e60 con 0x559d46707d60 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46701550 con 0x559d46706ff0 2024-03-11T19:24:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f889f5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46720280 con 0x559d46710570 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:60132/0 (socket says 172.21.15.72:60132) 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1815992334 learned_addr learned my addr 172.21.15.72:0/1815992334 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f7f400640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.121+0000 7f6f7e000640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1815992334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 msgr2=0x559d4670bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1815992334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 msgr2=0x559d4670dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 -- 172.21.15.72:0/1815992334 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d46608530 con 0x559d46706ff0 2024-03-11T19:24:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7e000640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f7009a210 tx=0x7f6f7000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5accb9e982519dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f6b400640 1 -- 172.21.15.72:0/1815992334 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7009ac10 con 0x559d46706ff0 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f6b400640 1 -- 172.21.15.72:0/1815992334 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f7007d070 con 0x559d46706ff0 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.122+0000 7f6f6b400640 1 -- 172.21.15.72:0/1815992334 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70005020 con 0x559d46706ff0 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.123+0000 7f6f889f5740 1 -- 172.21.15.72:0/1815992334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 msgr2=0x559d4670fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.123+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f7009a210 tx=0x7f6f7000c7d0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 -- 172.21.15.72:0/1815992334 shutdown_connections 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46710570 0x559d4670bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46707d60 0x559d4670dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 --2- 172.21.15.72:0/1815992334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d4670fd70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 -- 172.21.15.72:0/1815992334 >> 172.21.15.72:0/1815992334 conn(0x559d466ce4b0 msgr2=0x559d46708950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 -- 172.21.15.72:0/1815992334 shutdown_connections 2024-03-11T19:24:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.124+0000 7f6f889f5740 1 -- 172.21.15.72:0/1815992334 wait complete. 2024-03-11T19:24:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.125+0000 7f6f889f5740 1 -- start start 2024-03-11T19:24:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.125+0000 7f6f889f5740 1 Processor -- start 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46717070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46710570 0x559d46715580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46715c50 con 0x559d46710570 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46715dc0 con 0x559d46706ff0 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46715f80 con 0x559d46707d60 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f889f5740 10 client.0 _ll_register_callbacks cb 0x7f6f70096090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:55672/0 (socket says 172.21.15.72:55672) 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 -- 172.21.15.72:0/792345083 learned_addr learned my addr 172.21.15.72:0/792345083 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7f400640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46710570 0x559d46715580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 -- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 msgr2=0x559d46717070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46717070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 -- 172.21.15.72:0/792345083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46710570 msgr2=0x559d46715580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46710570 0x559d46715580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.126+0000 7f6f7e000640 1 -- 172.21.15.72:0/792345083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d4666bc50 con 0x559d46707d60 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.127+0000 7f6f7e000640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x559d46702e70 tx=0x7f6f6c010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b83420b5ecace4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.127+0000 7f6f68c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:24:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.127+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c015040 con 0x559d46707d60 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.127+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f6f6c00fe20 con 0x559d46707d60 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.127+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d46692d30 con 0x559d46707d60 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.128+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f6c01f0e0 con 0x559d46707d60 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.128+0000 7f6f889f5740 10 client.25561 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.128+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d4662e2c0 con 0x559d46707d60 2024-03-11T19:24:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.128+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d46692d30 con 0x559d46707d60 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f6c028470 con 0x559d46707d60 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 10 client.25561 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 20 client.25561 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46777af0 0x559d46779f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 4 client.25561 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.129+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d4636d600 con 0x559d46777af0 2024-03-11T19:24:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.130+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46777af0 0x559d46779f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.131+0000 7f6f7ea00640 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46777af0 0x559d46779f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68295f6b47d487 server_cookie=b3b8e60d912e2df1 in_seq=0 out_seq=0 2024-03-11T19:24:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.131+0000 7f6f68c00640 10 client.25561 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f68c00640 1 -- 172.21.15.72:0/792345083 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d4636d600 con 0x559d46777af0 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f68c00640 10 client.25561 handle_command_reply: tid=0 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 client.25561 shutdown 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 msgr2=0x559d467175b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x559d46702e70 tx=0x7f6f6c010d80 comp rx=0 tx=0).stop 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 shutdown_connections 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x559d46777af0 0x559d46779f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559d46710570 0x559d46715580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d46707d60 0x559d467175b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 --2- 172.21.15.72:0/792345083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d46706ff0 0x559d46717070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.135+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 >> 172.21.15.72:0/792345083 conn(0x559d466ce4b0 msgr2=0x559d4670be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.136+0000 7f6f68c00640 0 client.25561 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.136+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 shutdown_connections 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.136+0000 7f6f889f5740 1 -- 172.21.15.72:0/792345083 wait complete. 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.136+0000 7f6f889f5740 20 client.25561 trim_cache size 0 max 16384 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:24:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:24:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:24:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 msgr2=0x7f6f640a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 0x7f6f640a4120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f7802c430 tx=0x7f6f7802bef0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 msgr2=0x7f6f80101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 0x7f6f80101f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009fd0 tx=0x7f6f7000ec20 comp rx=0 tx=0).stop 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 shutdown_connections 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f6f640a1c60 0x7f6f640a4120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f8010d8f0 0x7f6f80102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f801006f0 0x7f6f801024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 --2- 172.21.15.72:0/713463982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f800ffd40 0x7f6f80101f90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 >> 172.21.15.72:0/713463982 conn(0x7f6f800fb990 msgr2=0x7f6f800fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 shutdown_connections 2024-03-11T19:24:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:18.137+0000 7f6f7fe00640 1 -- 172.21.15.72:0/713463982 wait complete. 2024-03-11T19:24:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:18 smithi106 ceph-mon[35889]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:18.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:18 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:24:18.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:18 smithi072 ceph-mon[31695]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:18 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:24:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:18 smithi143 ceph-mon[36013]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:24:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:18 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:24:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:19 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:24:19.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:19 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:24:19.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:19 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:24:19.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:19 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:24:19.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:19 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:24:19.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:19 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:24:19.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:19 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:24:19.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:19 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:24:19.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:19 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:24:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:20 smithi106 ceph-mon[35889]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:24:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:20 smithi072 ceph-mon[31695]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:24:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:20 smithi143 ceph-mon[36013]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:24:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:22 smithi106 ceph-mon[35889]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:24:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:22 smithi072 ceph-mon[31695]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:24:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:22 smithi143 ceph-mon[36013]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:24:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:24 smithi106 ceph-mon[35889]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:24.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:24 smithi072 ceph-mon[31695]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:24 smithi143 ceph-mon[36013]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:25.062 INFO:tasks.workunit.client.0.smithi072.stderr:Updating files: 64% (7951/12344) Updating files: 65% (8024/12344) Updating files: 66% (8148/12344) Updating files: 67% (8271/12344) Updating files: 68% (8394/12344) Updating files: 69% (8518/12344) Updating files: 70% (8641/12344) Updating files: 71% (8765/12344) Updating files: 72% (8888/12344) Updating files: 73% (9012/12344) Updating files: 74% (9135/12344) Updating files: 75% (9258/12344) Updating files: 76% (9382/12344) Updating files: 77% (9505/12344) Updating files: 78% (9629/12344) Updating files: 79% (9752/12344) Updating files: 80% (9876/12344) Updating files: 81% (9999/12344) Updating files: 82% (10123/12344) Updating files: 83% (10246/12344) Updating files: 84% (10369/12344) Updating files: 85% (10493/12344) Updating files: 86% (10616/12344) Updating files: 87% (10740/12344) Updating files: 88% (10863/12344) Updating files: 89% (10987/12344) Updating files: 90% (11110/12344) Updating files: 91% (11234/12344) Updating files: 92% (11357/12344) Updating files: 93% (11480/12344) Updating files: 94% (11604/12344) Updating files: 95% (11727/12344) Updating files: 96% (11851/12344) Updating files: 97% (11974/12344) Updating files: 98% (12098/12344) Updating files: 99% (12221/12344) Updating files: 100% (12344/12344) Updating files: 100% (12344/12344), done. 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:Note: switching to 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'. 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:state without impacting any branches by switching back to a branch. 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-11T19:24:25.350 INFO:tasks.workunit.client.0.smithi072.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: git switch -c 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr:Or undo this operation with: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: git switch - 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-03-11T19:24:25.351 INFO:tasks.workunit.client.0.smithi072.stderr:HEAD is now at b6a4997be2b Merge pull request #56060 from rhcs-dashboard/wip-64808-squid 2024-03-11T19:24:25.358 DEBUG:teuthology.orchestra.run.smithi072:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-11T19:24:25.464 INFO:tasks.workunit.client.0.smithi072.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-11T19:24:25.471 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T19:24:25.472 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-11T19:24:26.858 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-11T19:24:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:26 smithi106 ceph-mon[35889]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:26 smithi072 ceph-mon[31695]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:26 smithi143 ceph-mon[36013]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:27.033 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-11T19:24:27.117 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T19:24:27.122 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T19:24:27.122 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-11T19:24:27.200 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T19:24:27.211 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:24:27.211 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-11T19:24:27.259 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-03-11T19:24:27.261 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-03-11T19:24:27.261 DEBUG:teuthology.orchestra.run.smithi072:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-03-11T19:24:27.372 INFO:tasks.workunit.client.0.smithi072.stdout:test large (16MB) dio write 2024-03-11T19:24:27.373 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo 'test large (16MB) dio write' 2024-03-11T19:24:27.373 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-03-11T19:24:27.694 INFO:tasks.workunit.client.0.smithi072.stderr:1+0 records in 2024-03-11T19:24:27.694 INFO:tasks.workunit.client.0.smithi072.stderr:1+0 records out 2024-03-11T19:24:27.694 INFO:tasks.workunit.client.0.smithi072.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.314753 s, 53.3 MB/s 2024-03-11T19:24:27.697 INFO:tasks.workunit.client.0.smithi072.stdout:OK 2024-03-11T19:24:27.698 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo OK 2024-03-11T19:24:27.699 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:24:27.699 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:24:27.815 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-03-11T19:24:27.815 DEBUG:teuthology.orchestra.run.smithi072:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-03-11T19:24:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:28 smithi106 ceph-mon[35889]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:24:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:28 smithi072 ceph-mon[31695]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:24:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:28 smithi143 ceph-mon[36013]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:24:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:30 smithi106 ceph-mon[35889]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:30 smithi072 ceph-mon[31695]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:30 smithi143 ceph-mon[36013]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:24:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 -- 172.21.15.72:0/3204292755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea70103d80 msgr2=0x7fea70107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 --2- 172.21.15.72:0/3204292755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea70103d80 0x7fea70107dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:24:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 -- 172.21.15.72:0/3204292755 shutdown_connections 2024-03-11T19:24:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 --2- 172.21.15.72:0/3204292755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 0x7fea70110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 --2- 172.21.15.72:0/3204292755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea70103d80 0x7fea70107dc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 --2- 172.21.15.72:0/3204292755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 0x7fea701037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 -- 172.21.15.72:0/3204292755 >> 172.21.15.72:0/3204292755 conn(0x7fea700fec90 msgr2=0x7fea701010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 -- 172.21.15.72:0/3204292755 shutdown_connections 2024-03-11T19:24:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.285+0000 7fea76a00640 1 -- 172.21.15.72:0/3204292755 wait complete. 2024-03-11T19:24:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.286+0000 7fea76a00640 1 Processor -- start 2024-03-11T19:24:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 -- start start 2024-03-11T19:24:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 0x7fea7019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 0x7fea7019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019a170 con 0x7fea701084a0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019a2e0 con 0x7fea701033d0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019a4a0 con 0x7fea701996a0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea74c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 0x7fea7019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:42868/0 (socket says 172.21.15.72:42868) 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea76000640 1 -- 172.21.15.72:0/557805468 learned_addr learned my addr 172.21.15.72:0/557805468 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.287+0000 7fea75600640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 0x7fea7019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 -- 172.21.15.72:0/557805468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 msgr2=0x7fea7019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 0x7fea7019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 -- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 msgr2=0x7fea7019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 0x7fea7019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 -- 172.21.15.72:0/557805468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68009840 con 0x7fea701996a0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76000640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea6800efd0 tx=0x7fea6800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea6e000640 1 -- 172.21.15.72:0/557805468 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68009b40 con 0x7fea701996a0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c009560 con 0x7fea701996a0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea6e000640 1 -- 172.21.15.72:0/557805468 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea68003b70 con 0x7fea701996a0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea6e000640 1 -- 172.21.15.72:0/557805468 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea680108f0 con 0x7fea701996a0 2024-03-11T19:24:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.288+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea701a6740 con 0x7fea701996a0 2024-03-11T19:24:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea6e000640 1 -- 172.21.15.72:0/557805468 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 136512+0+0 (secure 0 0 0) 0x7fea68003ce0 con 0x7fea701996a0 2024-03-11T19:24:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 Processor -- start 2024-03-11T19:24:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 -- start start 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fb7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10f73620 con 0x555c10fc1570 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10f258c0 con 0x555c10fb7ff0 2024-03-11T19:24:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.292+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10f22df0 con 0x555c10fb8d60 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:53302/0 (socket says 172.21.15.72:53302) 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 -- 172.21.15.72:0/2819782741 learned_addr learned my addr 172.21.15.72:0/2819782741 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea74c00640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fb7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea74c00640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 -- 172.21.15.72:0/2819782741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 msgr2=0x555c10fb7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fb7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 -- 172.21.15.72:0/2819782741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 msgr2=0x555c10fc3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea75600640 1 -- 172.21.15.72:0/2819782741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c10e9fb00 con 0x555c10fb7ff0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.293+0000 7fea74c00640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fb7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea75600640 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea64009c90 tx=0x7fea640117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13129ff88b3ebc8c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea6e000640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 0x7fea580a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea62600640 1 -- 172.21.15.72:0/2819782741 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64009760 con 0x555c10fb7ff0 2024-03-11T19:24:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea62600640 1 -- 172.21.15.72:0/2819782741 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea64012930 con 0x555c10fb7ff0 2024-03-11T19:24:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea62600640 1 -- 172.21.15.72:0/2819782741 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6400fb40 con 0x555c10fb7ff0 2024-03-11T19:24:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea6e000640 1 -- 172.21.15.72:0/557805468 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fea68014070 con 0x7fea701996a0 2024-03-11T19:24:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea74c00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 0x7fea580a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea7ad8f740 1 -- 172.21.15.72:0/2819782741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 msgr2=0x555c10fb8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.294+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea64009c90 tx=0x7fea640117b0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 -- 172.21.15.72:0/2819782741 shutdown_connections 2024-03-11T19:24:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fb8470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/2819782741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fb7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 -- 172.21.15.72:0/2819782741 >> 172.21.15.72:0/2819782741 conn(0x555c10f7f4b0 msgr2=0x555c10fb2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea74c00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 0x7fea580a4170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009230 tx=0x7fea5c005fd0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 -- 172.21.15.72:0/2819782741 shutdown_connections 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 -- 172.21.15.72:0/2819782741 wait complete. 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 -- start start 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.295+0000 7fea7ad8f740 1 Processor -- start 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb7ff0 0x555c10fbe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fbea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10f1e5d0 con 0x555c10fb7ff0 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10ec3b50 con 0x555c10fc1570 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10ec37f0 con 0x555c10fb8d60 2024-03-11T19:24:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea7ad8f740 10 client.0 _ll_register_callbacks cb 0x555c10ed7550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea76000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea74c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fbea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea76000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:53318/0 (socket says 172.21.15.72:53318) 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea76000640 1 -- 172.21.15.72:0/4197953484 learned_addr learned my addr 172.21.15.72:0/4197953484 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea75600640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb7ff0 0x555c10fbe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.296+0000 7fea76000640 1 -- 172.21.15.72:0/4197953484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 msgr2=0x555c10fbea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea76000640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fbea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea76000640 1 -- 172.21.15.72:0/4197953484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb7ff0 msgr2=0x555c10fbe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea76000640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb7ff0 0x555c10fbe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea76000640 1 -- 172.21.15.72:0/4197953484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea640098c0 con 0x555c10fc1570 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea76000640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fea6809a9c0 tx=0x7fea6809a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434fd2cfcdf46a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea56a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea680cbb00 con 0x555c10fc1570 2024-03-11T19:24:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fea680a31d0 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c10ef0cd0 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea68004b40 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea7ad8f740 10 client.25580 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c10f1cc50 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.297+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c10ef0cd0 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea68096780 con 0x555c10fc1570 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 10 client.25580 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 20 client.25580 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c10fd1530 0x555c10fd3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 4 client.25580 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c10f14ce0 con 0x555c10fd1530 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.298+0000 7fea75600640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c10fd1530 0x555c10fd3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.299+0000 7fea75600640 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c10fd1530 0x555c10fd3920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ca1e53e7f430c1 server_cookie=609dd86ab360b9a2 in_seq=0 out_seq=0 2024-03-11T19:24:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.299+0000 7fea56a00640 10 client.25580 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.299+0000 7fea56a00640 1 -- 172.21.15.72:0/4197953484 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c10f14ce0 con 0x555c10fd1530 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.299+0000 7fea56a00640 10 client.25580 handle_command_reply: tid=0 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 client.25580 shutdown 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 msgr2=0x555c10fbef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fea6809a9c0 tx=0x7fea6809a480 comp rx=0 tx=0).stop 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 shutdown_connections 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c10fd1530 0x555c10fd3920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fc1570 0x555c10fbef90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fbea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/4197953484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb7ff0 0x555c10fbe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea56a00640 0 client.25580 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 >> 172.21.15.72:0/4197953484 conn(0x555c10f7f4b0 msgr2=0x555c10fb4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 shutdown_connections 2024-03-11T19:24:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 1 -- 172.21.15.72:0/4197953484 wait complete. 2024-03-11T19:24:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.300+0000 7fea7ad8f740 20 client.25580 trim_cache size 0 max 16384 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.315+0000 7fea7ad8f740 1 Processor -- start 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 -- start start 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb8d60 0x555c10fb9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10c66e60 con 0x555c10fb8d60 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10fb72c0 con 0x555c10fb7ff0 2024-03-11T19:24:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10fcd980 con 0x555c10fc1570 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea76000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea76000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:42894/0 (socket says 172.21.15.72:42894) 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea76000640 1 -- 172.21.15.72:0/1129412600 learned_addr learned my addr 172.21.15.72:0/1129412600 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea74c00640 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb8d60 0x555c10fb9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.316+0000 7fea75600640 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 -- 172.21.15.72:0/1129412600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 msgr2=0x555c10fc0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 -- 172.21.15.72:0/1129412600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb8d60 msgr2=0x555c10fb9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb8d60 0x555c10fb9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 -- 172.21.15.72:0/1129412600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c10eb9530 con 0x555c10fc1570 2024-03-11T19:24:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea76000640 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea680973b0 tx=0x7fea6809ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9892b8f52f3d17c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea61c00640 1 -- 172.21.15.72:0/1129412600 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68066230 con 0x555c10fc1570 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea61c00640 1 -- 172.21.15.72:0/1129412600 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea680a31d0 con 0x555c10fc1570 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.317+0000 7fea61c00640 1 -- 172.21.15.72:0/1129412600 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68004b40 con 0x555c10fc1570 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1129412600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 msgr2=0x555c10fbedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea680973b0 tx=0x7fea6809ed00 comp rx=0 tx=0).stop 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1129412600 shutdown_connections 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fc1570 0x555c10fbedb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fb8d60 0x555c10fb9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1129412600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1129412600 >> 172.21.15.72:0/1129412600 conn(0x555c10f7f4b0 msgr2=0x555c10fbcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1129412600 shutdown_connections 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1129412600 wait complete. 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.318+0000 7fea7ad8f740 1 -- start start 2024-03-11T19:24:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 Processor -- start 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10fc7420 con 0x555c10fc1570 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10fc7590 con 0x555c10fb7ff0 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10fc7750 con 0x555c10fb8d60 2024-03-11T19:24:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea7ad8f740 10 client.0 _ll_register_callbacks cb 0x7fea68004880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea74c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea76000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea74c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:42900/0 (socket says 172.21.15.72:42900) 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.319+0000 7fea74c00640 1 -- 172.21.15.72:0/1484471699 learned_addr learned my addr 172.21.15.72:0/1484471699 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea75600640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 -- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 msgr2=0x555c10fc8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 -- 172.21.15.72:0/1484471699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 msgr2=0x555c10fc6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 -- 172.21.15.72:0/1484471699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c10f1cc50 con 0x555c10fb8d60 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea76000640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea74c00640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd98c1459911d1f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea56a00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c004df0 con 0x555c10fb8d60 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c10f43d30 con 0x555c10fb8d60 2024-03-11T19:24:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.320+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fea5c04b040 con 0x555c10fb8d60 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5c004f50 con 0x555c10fb8d60 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 10 client.25579 fetch_fsmap learned FSMap version 25 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c10edf2c0 con 0x555c10fb8d60 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea5c064020 con 0x555c10fb8d60 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea5c03c040 con 0x555c10fb8d60 2024-03-11T19:24:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 10 client.25579 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 20 client.25579 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c110290b0 0x555c1102b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 4 client.25579 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c10c1e600 con 0x555c110290b0 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.321+0000 7fea75600640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c110290b0 0x555c1102b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.322+0000 7fea75600640 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c110290b0 0x555c1102b4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbfc8d9b139ba5d server_cookie=ba34343f89d6650b in_seq=0 out_seq=0 2024-03-11T19:24:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.322+0000 7fea56a00640 10 client.25579 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea56a00640 1 -- 172.21.15.72:0/1484471699 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c10c1e600 con 0x555c110290b0 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea56a00640 10 client.25579 handle_command_reply: tid=0 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 client.25579 shutdown 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 msgr2=0x555c10fc69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009900 tx=0x7fea5c005e80 comp rx=0 tx=0).stop 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 shutdown_connections 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x555c110290b0 0x555c1102b4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555c10fc1570 0x555c10fc6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c10fb8d60 0x555c10fc69a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea56a00640 0 client.25579 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 --2- 172.21.15.72:0/1484471699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555c10fb7ff0 0x555c10fc8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.334+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 >> 172.21.15.72:0/1484471699 conn(0x555c10f7f4b0 msgr2=0x555c10fb14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 shutdown_connections 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea7ad8f740 1 -- 172.21.15.72:0/1484471699 wait complete. 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea7ad8f740 20 client.25579 trim_cache size 0 max 16384 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:24:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 msgr2=0x7fea580a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 0x7fea580a4170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009230 tx=0x7fea5c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 msgr2=0x7fea70199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.335+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fea6800efd0 tx=0x7fea6800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 shutdown_connections 2024-03-11T19:24:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fea580a1cb0 0x7fea580a4170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea701996a0 0x7fea70199b20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea701084a0 0x7fea7019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 --2- 172.21.15.72:0/557805468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea701033d0 0x7fea7019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 >> 172.21.15.72:0/557805468 conn(0x7fea700fec90 msgr2=0x7fea70100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 shutdown_connections 2024-03-11T19:24:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:31.336+0000 7fea76a00640 1 -- 172.21.15.72:0/557805468 wait complete. 2024-03-11T19:24:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:32 smithi106 ceph-mon[35889]: pgmap v44: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-03-11T19:24:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:32 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:32 smithi072 ceph-mon[31695]: pgmap v44: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-03-11T19:24:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:32 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:32 smithi143 ceph-mon[36013]: pgmap v44: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.4 MiB/s wr, 215 op/s 2024-03-11T19:24:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:32 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:34 smithi106 ceph-mon[35889]: pgmap v45: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:34 smithi072 ceph-mon[31695]: pgmap v45: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:34 smithi143 ceph-mon[36013]: pgmap v45: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:36 smithi106 ceph-mon[35889]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:36 smithi072 ceph-mon[31695]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:36 smithi143 ceph-mon[36013]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-11T19:24:37.004 INFO:tasks.workunit.client.0.smithi072.stdout:.......... 2024-03-11T19:24:37.004 INFO:tasks.workunit.client.0.smithi072.stdout:do_o_direct_reads: SUCCESS 2024-03-11T19:24:37.008 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:24:37.008 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:24:37.116 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-03-11T19:24:37.116 DEBUG:teuthology.orchestra.run.smithi072:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-03-11T19:24:37.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:37 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:37 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:37 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:24:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:38 smithi106 ceph-mon[35889]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-03-11T19:24:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:38 smithi072 ceph-mon[31695]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-03-11T19:24:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:38 smithi143 ceph-mon[36013]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-03-11T19:24:39.621 INFO:tasks.workunit.client.0.smithi072.stdout:test read from hole 2024-03-11T19:24:39.621 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo 'test read from hole' 2024-03-11T19:24:39.621 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-03-11T19:24:39.626 INFO:tasks.workunit.client.0.smithi072.stderr:0+0 records in 2024-03-11T19:24:39.626 INFO:tasks.workunit.client.0.smithi072.stderr:0+0 records out 2024-03-11T19:24:39.626 INFO:tasks.workunit.client.0.smithi072.stderr:0 bytes copied, 9.4121e-05 s, 0.0 kB/s 2024-03-11T19:24:39.627 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-03-11T19:24:39.633 INFO:tasks.workunit.client.0.smithi072.stderr:2+0 records in 2024-03-11T19:24:39.634 INFO:tasks.workunit.client.0.smithi072.stderr:2+0 records out 2024-03-11T19:24:39.634 INFO:tasks.workunit.client.0.smithi072.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00346434 s, 296 kB/s 2024-03-11T19:24:39.634 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-03-11T19:24:39.637 INFO:tasks.workunit.client.0.smithi072.stderr:2+0 records in 2024-03-11T19:24:39.637 INFO:tasks.workunit.client.0.smithi072.stderr:2+0 records out 2024-03-11T19:24:39.637 INFO:tasks.workunit.client.0.smithi072.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000126142 s, 8.1 MB/s 2024-03-11T19:24:39.637 INFO:tasks.workunit.client.0.smithi072.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-03-11T19:24:39.640 INFO:tasks.workunit.client.0.smithi072.stdout:other thing 2024-03-11T19:24:39.641 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo 'other thing' 2024-03-11T19:24:39.641 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-03-11T19:24:39.643 INFO:tasks.workunit.client.0.smithi072.stderr:1+0 records in 2024-03-11T19:24:39.644 INFO:tasks.workunit.client.0.smithi072.stderr:1+0 records out 2024-03-11T19:24:39.644 INFO:tasks.workunit.client.0.smithi072.stderr:500 bytes copied, 0.000124722 s, 4.0 MB/s 2024-03-11T19:24:39.644 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-03-11T19:24:39.657 INFO:tasks.workunit.client.0.smithi072.stderr:0+1 records in 2024-03-11T19:24:39.657 INFO:tasks.workunit.client.0.smithi072.stderr:0+1 records out 2024-03-11T19:24:39.657 INFO:tasks.workunit.client.0.smithi072.stderr:500 bytes copied, 0.0103496 s, 48.3 kB/s 2024-03-11T19:24:39.657 INFO:tasks.workunit.client.0.smithi072.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-03-11T19:24:39.671 INFO:tasks.workunit.client.0.smithi072.stderr:0+1 records in 2024-03-11T19:24:39.671 INFO:tasks.workunit.client.0.smithi072.stderr:0+1 records out 2024-03-11T19:24:39.671 INFO:tasks.workunit.client.0.smithi072.stderr:500 bytes copied, 0.00972835 s, 51.4 kB/s 2024-03-11T19:24:39.671 INFO:tasks.workunit.client.0.smithi072.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-03-11T19:24:39.675 INFO:tasks.workunit.client.0.smithi072.stdout:OK 2024-03-11T19:24:39.675 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo OK 2024-03-11T19:24:39.676 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:24:39.677 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:24:39.787 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-03-11T19:24:39.787 DEBUG:teuthology.orchestra.run.smithi072:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-03-11T19:24:39.880 INFO:tasks.workunit.client.0.smithi072.stdout:writing first 3 bytes of 10k file 2024-03-11T19:24:39.880 INFO:tasks.workunit.client.0.smithi072.stdout:reading O_DIRECT 2024-03-11T19:24:39.880 INFO:tasks.workunit.client.0.smithi072.stdout:got 10000 2024-03-11T19:24:39.881 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:24:39.882 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:24:39.996 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-03-11T19:24:39.997 DEBUG:teuthology.orchestra.run.smithi072:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-03-11T19:24:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:40 smithi106 ceph-mon[35889]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 643 op/s 2024-03-11T19:24:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:40 smithi072 ceph-mon[31695]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 643 op/s 2024-03-11T19:24:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:40 smithi143 ceph-mon[36013]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 643 op/s 2024-03-11T19:24:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:42 smithi106 ceph-mon[35889]: pgmap v49: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 788 op/s 2024-03-11T19:24:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:42 smithi106 ceph-mon[35889]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-11T19:24:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:42 smithi072 ceph-mon[31695]: pgmap v49: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 788 op/s 2024-03-11T19:24:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:42 smithi072 ceph-mon[31695]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-11T19:24:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:42 smithi143 ceph-mon[36013]: pgmap v49: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 788 op/s 2024-03-11T19:24:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:42 smithi143 ceph-mon[36013]: mgrmap e28: x(active, since 92s), standbys: y, z 2024-03-11T19:24:43.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:43 smithi106 ceph-mon[35889]: pgmap v50: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:43.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:43 smithi072 ceph-mon[31695]: pgmap v50: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:43.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:43 smithi143 ceph-mon[36013]: pgmap v50: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 -- 172.21.15.72:0/1308481859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81011d0 msgr2=0x7fb0c81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 --2- 172.21.15.72:0/1308481859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81011d0 0x7fb0c81015b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009930 tx=0x7fb0b802bc20 comp rx=0 tx=0).stop 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 -- 172.21.15.72:0/1308481859 shutdown_connections 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 --2- 172.21.15.72:0/1308481859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81062a0 0x7fb0c810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 --2- 172.21.15.72:0/1308481859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8101b80 0x7fb0c8105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 --2- 172.21.15.72:0/1308481859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81011d0 0x7fb0c81015b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 -- 172.21.15.72:0/1308481859 >> 172.21.15.72:0/1308481859 conn(0x7fb0c80fca90 msgr2=0x7fb0c80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 -- 172.21.15.72:0/1308481859 shutdown_connections 2024-03-11T19:24:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.468+0000 7fb0cda00640 1 -- 172.21.15.72:0/1308481859 wait complete. 2024-03-11T19:24:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.469+0000 7fb0cda00640 1 Processor -- start 2024-03-11T19:24:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.469+0000 7fb0cda00640 1 -- start start 2024-03-11T19:24:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.469+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81011d0 0x7fb0c819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.469+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.469+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8199730 0x7fb0c8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0cda00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c819a200 con 0x7fb0c81062a0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0cda00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c819a370 con 0x7fb0c81011d0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0cda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c819a530 con 0x7fb0c8199730 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52078/0 (socket says 172.21.15.72:52078) 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 -- 172.21.15.72:0/739577250 learned_addr learned my addr 172.21.15.72:0/739577250 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81011d0 0x7fb0c819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0cd000640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8199730 0x7fb0c8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 -- 172.21.15.72:0/739577250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8199730 msgr2=0x7fb0c8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8199730 0x7fb0c8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 -- 172.21.15.72:0/739577250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81011d0 msgr2=0x7fb0c819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81011d0 0x7fb0c819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.470+0000 7fb0c7400640 1 -- 172.21.15.72:0/739577250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009560 con 0x7fb0c81062a0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0c7400640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e9f0 tx=0x7fb0b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0c4c00640 1 -- 172.21.15.72:0/739577250 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40096c0 con 0x7fb0c81062a0 2024-03-11T19:24:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0c4c00640 1 -- 172.21.15.72:0/739577250 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b4005500 con 0x7fb0c81062a0 2024-03-11T19:24:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c81a64d0 con 0x7fb0c81062a0 2024-03-11T19:24:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c81a6960 con 0x7fb0c81062a0 2024-03-11T19:24:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.471+0000 7fb0c4c00640 1 -- 172.21.15.72:0/739577250 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4018430 con 0x7fb0c81062a0 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 Processor -- start 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 -- start start 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd89ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cd45620 con 0x558c9cd89ff0 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ccf78c0 con 0x558c9cd8ad60 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ccf4df0 con 0x558c9cd93570 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.474+0000 7fb0c4c00640 1 -- 172.21.15.72:0/739577250 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb0b4005000 con 0x7fb0c81062a0 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52092/0 (socket says 172.21.15.72:52092) 2024-03-11T19:24:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2343008612 learned_addr learned my addr 172.21.15.72:0/2343008612 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0cd000640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd89ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7400640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2343008612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 msgr2=0x558c9cd95960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd95960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2343008612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 msgr2=0x558c9cd89ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd89ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2343008612 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9cc71b00 con 0x558c9cd89ff0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0cd000640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd89ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.475+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8036040 tx=0x7fb0b802c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0e231f641307d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0bd600640 1 -- 172.21.15.72:0/2343008612 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8004800 con 0x558c9cd89ff0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0bd600640 1 -- 172.21.15.72:0/2343008612 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8004960 con 0x558c9cd89ff0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0bd600640 1 -- 172.21.15.72:0/2343008612 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8005120 con 0x558c9cd89ff0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0c4c00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 0x7fb0ac0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0c4c00640 1 -- 172.21.15.72:0/739577250 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb0b4014070 con 0x7fb0c81062a0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0cd000640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 0x7fb0ac0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2343008612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 msgr2=0x558c9cd8a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.476+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8036040 tx=0x7fb0b802c5c0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2343008612 shutdown_connections 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd8a470 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2343008612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd89ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2343008612 >> 172.21.15.72:0/2343008612 conn(0x558c9cd514b0 msgr2=0x558c9cd84ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2343008612 shutdown_connections 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0cd000640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 0x7fb0ac0a4050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fd0 tx=0x7fb0c0009500 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2343008612 wait complete. 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 -- start start 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.477+0000 7fb0d1d76740 1 Processor -- start 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd91060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ccf05d0 con 0x558c9cd89ff0 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cc95b50 con 0x558c9cd8ad60 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cc957f0 con 0x558c9cd93570 2024-03-11T19:24:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0d1d76740 10 client.0 _ll_register_callbacks cb 0x558c9cca9550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52106/0 (socket says 172.21.15.72:52106) 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0c7e00640 1 -- 172.21.15.72:0/3955488289 learned_addr learned my addr 172.21.15.72:0/3955488289 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0cd000640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd91060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.478+0000 7fb0c7e00640 1 -- 172.21.15.72:0/3955488289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 msgr2=0x558c9cd91060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd91060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7e00640 1 -- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 msgr2=0x558c9cd90b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7e00640 1 -- 172.21.15.72:0/3955488289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c810e680 con 0x558c9cd89ff0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0cd000640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd91060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802c160 tx=0x7fb0b802c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3716974e7bf5a3ff server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.487 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0b2000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b804f070 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb0b802cbb0 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c9ccc2cd0 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b8005070 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0d1d76740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.479+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c9cceec50 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558c9ccc2cd0 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb0b8052020 con 0x558c9cd89ff0 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 20 client.15501 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cda3220 0x558c9cda5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 4 client.15501 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:24:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c9cce6ce0 con 0x558c9cda3220 2024-03-11T19:24:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.480+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cda3220 0x558c9cda5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.481+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cda3220 0x558c9cda5660 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211267367a231cd4 server_cookie=a250807a9d068698 in_seq=0 out_seq=0 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.481+0000 7fb0b2000640 10 client.15501 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.481+0000 7fb0b2000640 1 -- 172.21.15.72:0/3955488289 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c9cce6ce0 con 0x558c9cda3220 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.481+0000 7fb0b2000640 10 client.15501 handle_command_reply: tid=0 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.481+0000 7fb0d1d76740 1 client.15501 shutdown 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 msgr2=0x558c9cd905e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802c160 tx=0x7fb0b802c730 comp rx=0 tx=0).stop 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 shutdown_connections 2024-03-11T19:24:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cda3220 0x558c9cda5660 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd93570 0x558c9cd91060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3955488289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd89ff0 0x558c9cd905e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 >> 172.21.15.72:0/3955488289 conn(0x558c9cd514b0 msgr2=0x558c9cd86a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0b2000640 0 client.15501 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 shutdown_connections 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3955488289 wait complete. 2024-03-11T19:24:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.482+0000 7fb0d1d76740 20 client.15501 trim_cache size 0 max 16384 2024-03-11T19:24:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 Processor -- start 2024-03-11T19:24:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 -- start start 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd93570 0x558c9cd9a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9ca38e60 con 0x558c9cd93570 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cd91930 con 0x558c9cd8ad60 2024-03-11T19:24:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.496+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cd80720 con 0x558c9cd89ff0 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:59324/0 (socket says 172.21.15.72:59324) 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0cd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd93570 0x558c9cd9a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2059094406 learned_addr learned my addr 172.21.15.72:0/2059094406 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7400640 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2059094406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 msgr2=0x558c9cd90e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2059094406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd93570 msgr2=0x558c9cd9a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd93570 0x558c9cd9a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 -- 172.21.15.72:0/2059094406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9cc8b530 con 0x558c9cd89ff0 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7400640 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.497+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005900 tx=0x7fb0b802bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683f2cbef62a5f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0bcc00640 1 -- 172.21.15.72:0/2059094406 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b804f070 con 0x558c9cd89ff0 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0bcc00640 1 -- 172.21.15.72:0/2059094406 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b802c9e0 con 0x558c9cd89ff0 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0bcc00640 1 -- 172.21.15.72:0/2059094406 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8034440 con 0x558c9cd89ff0 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2059094406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 msgr2=0x558c9cd92d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005900 tx=0x7fb0b802bc20 comp rx=0 tx=0).stop 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2059094406 shutdown_connections 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd93570 0x558c9cd9a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd8ad60 0x558c9cd90e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/2059094406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd92d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2059094406 >> 172.21.15.72:0/2059094406 conn(0x558c9cd514b0 msgr2=0x558c9cda9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2059094406 shutdown_connections 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.498+0000 7fb0d1d76740 1 -- 172.21.15.72:0/2059094406 wait complete. 2024-03-11T19:24:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 -- start start 2024-03-11T19:24:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 Processor -- start 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd93570 0x558c9cdf8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cd97080 con 0x558c9cd8ad60 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cdf8c70 con 0x558c9cd93570 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9cdf8db0 con 0x558c9cd89ff0 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0d1d76740 10 client.0 _ll_register_callbacks cb 0x7fb0c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.499+0000 7fb0c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0cd000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd93570 0x558c9cdf8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52118/0 (socket says 172.21.15.72:52118) 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 -- 172.21.15.72:0/3836228934 learned_addr learned my addr 172.21.15.72:0/3836228934 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 -- 172.21.15.72:0/3836228934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 msgr2=0x558c9cd96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd96310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 -- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd93570 msgr2=0x558c9cdf8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd93570 0x558c9cdf8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 -- 172.21.15.72:0/3836228934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c81a6960 con 0x558c9cd8ad60 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0c7400640 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c6a0 tx=0x7fb0b40d17e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92704edf4cd58cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0b2000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40b2070 con 0x558c9cd8ad60 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb0b409aff0 con 0x558c9cd8ad60 2024-03-11T19:24:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.500+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c9cd15d30 con 0x558c9cd8ad60 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b4005670 con 0x558c9cd8ad60 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 10 client.15507 fetch_fsmap learned FSMap version 27 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c9cceec50 con 0x558c9cd8ad60 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0b407d020 con 0x558c9cd8ad60 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb0b401f060 con 0x558c9cd8ad60 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 20 client.15507 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cdfa390 0x558c9cdfc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 4 client.15507 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c9c9f0600 con 0x558c9cdfa390 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.501+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cdfa390 0x558c9cdfc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.502+0000 7fb0b2000640 10 client.15507 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.502+0000 7fb0c7e00640 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cdfa390 0x558c9cdfc7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4251ab04f95eb server_cookie=f1a4b3da21a85504 in_seq=0 out_seq=0 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0b2000640 1 -- 172.21.15.72:0/3836228934 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c9c9f0600 con 0x558c9cdfa390 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0b2000640 10 client.15507 handle_command_reply: tid=0 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 client.15507 shutdown 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 msgr2=0x558c9cd96850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c6a0 tx=0x7fb0b40d17e0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 shutdown_connections 2024-03-11T19:24:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x558c9cdfa390 0x558c9cdfc7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558c9cd93570 0x558c9cdf8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558c9cd8ad60 0x558c9cd96850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0b2000640 0 client.15507 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 --2- 172.21.15.72:0/3836228934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c9cd89ff0 0x558c9cd96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.511+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 >> 172.21.15.72:0/3836228934 conn(0x558c9cd514b0 msgr2=0x558c9cd84810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.512+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 shutdown_connections 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.512+0000 7fb0d1d76740 1 -- 172.21.15.72:0/3836228934 wait complete. 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.512+0000 7fb0d1d76740 20 client.15507 trim_cache size 0 max 16384 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:24:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:24:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:24:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.512+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 msgr2=0x7fb0ac0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.512+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 0x7fb0ac0a4050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fd0 tx=0x7fb0c0009500 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 msgr2=0x7fb0c819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e9f0 tx=0x7fb0b400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 shutdown_connections 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb0ac0a1b90 0x7fb0ac0a4050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0c8199730 0x7fb0c8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c81062a0 0x7fb0c819fbb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 --2- 172.21.15.72:0/739577250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c81011d0 0x7fb0c819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 >> 172.21.15.72:0/739577250 conn(0x7fb0c80fca90 msgr2=0x7fb0c80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 shutdown_connections 2024-03-11T19:24:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:44.513+0000 7fb0cda00640 1 -- 172.21.15.72:0/739577250 wait complete. 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:writing pattern 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-11T19:24:45.179 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:45.180 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:45.181 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:45.182 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:45.183 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:45.184 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:45.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:45.286 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:45.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:45.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:45.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:45.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:45.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:45.343 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:45.343 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:45.343 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:45.344 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:45.345 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:45.346 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:45.347 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:45.348 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:45.349 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:45.405 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:45.406 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:45.407 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:45.408 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-11T19:24:45.409 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:45.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:45.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:45.506 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:45.507 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:45.508 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:45.509 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:45.510 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:45.511 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:45.512 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:45.589 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:45.590 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:45.591 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:45.592 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:45.593 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:45.594 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:45.633 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:45.633 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:45.633 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:45.634 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:45.635 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-03-11T19:24:45.636 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-03-11T19:24:45.637 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-11T19:24:45.638 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-03-11T19:24:45.639 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-11T19:24:45.639 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-11T19:24:45.703 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-03-11T19:24:45.704 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-03-11T19:24:45.705 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-03-11T19:24:45.706 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-11T19:24:45.707 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-03-11T19:24:45.708 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-03-11T19:24:45.790 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-11T19:24:45.791 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-03-11T19:24:45.792 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-11T19:24:45.793 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-03-11T19:24:45.794 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-11T19:24:45.795 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-11T19:24:45.861 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-03-11T19:24:45.862 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-03-11T19:24:45.863 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-03-11T19:24:45.864 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-11T19:24:45.865 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-03-11T19:24:45.866 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-11T19:24:45.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-03-11T19:24:45.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-11T19:24:45.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-03-11T19:24:45.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-11T19:24:45.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-11T19:24:45.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-11T19:24:45.908 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-11T19:24:45.909 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-03-11T19:24:45.910 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-03-11T19:24:45.911 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-11T19:24:45.912 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-11T19:24:45.913 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-03-11T19:24:45.914 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-11T19:24:45.914 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-03-11T19:24:45.914 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-11T19:24:45.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-11T19:24:45.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-03-11T19:24:45.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-03-11T19:24:45.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-03-11T19:24:45.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-11T19:24:45.982 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-03-11T19:24:45.982 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-11T19:24:45.982 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-03-11T19:24:45.982 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-11T19:24:46.070 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-11T19:24:46.071 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-03-11T19:24:46.072 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-11T19:24:46.073 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-03-11T19:24:46.074 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-11T19:24:46.075 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-03-11T19:24:46.076 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-11T19:24:46.136 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-11T19:24:46.137 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-03-11T19:24:46.138 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-11T19:24:46.139 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-03-11T19:24:46.140 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-11T19:24:46.141 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-11T19:24:46.142 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-11T19:24:46.185 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-03-11T19:24:46.186 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-03-11T19:24:46.187 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-11T19:24:46.188 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-03-11T19:24:46.189 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-03-11T19:24:46.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-11T19:24:46.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-11T19:24:46.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-03-11T19:24:46.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-03-11T19:24:46.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-03-11T19:24:46.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-11T19:24:46.196 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-03-11T19:24:46.196 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-11T19:24:46.196 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-03-11T19:24:46.196 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-11T19:24:46.196 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-11T19:24:46.257 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-03-11T19:24:46.258 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-03-11T19:24:46.259 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-03-11T19:24:46.260 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-11T19:24:46.261 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-03-11T19:24:46.262 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-03-11T19:24:46.350 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-11T19:24:46.351 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-11T19:24:46.352 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-03-11T19:24:46.353 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-03-11T19:24:46.354 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-03-11T19:24:46.355 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-11T19:24:46.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-03-11T19:24:46.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-11T19:24:46.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-03-11T19:24:46.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-11T19:24:46.410 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-11T19:24:46.411 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-11T19:24:46.412 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-03-11T19:24:46.413 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-03-11T19:24:46.414 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-11T19:24:46.415 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-03-11T19:24:46.416 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-11T19:24:46.457 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-03-11T19:24:46.458 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-11T19:24:46.459 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-03-11T19:24:46.460 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-11T19:24:46.461 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-11T19:24:46.462 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-03-11T19:24:46.549 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-03-11T19:24:46.550 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-11T19:24:46.551 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-11T19:24:46.552 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-03-11T19:24:46.553 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-11T19:24:46.554 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-03-11T19:24:46.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:46 smithi143 ceph-mon[36013]: pgmap v51: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:46.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:46 smithi106 ceph-mon[35889]: pgmap v51: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:46.658 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-11T19:24:46.658 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-03-11T19:24:46.658 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-11T19:24:46.659 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-03-11T19:24:46.660 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-11T19:24:46.661 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-11T19:24:46.662 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-11T19:24:46.663 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-03-11T19:24:46.664 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-11T19:24:46.664 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-03-11T19:24:46.664 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-11T19:24:46.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:46 smithi072 ceph-mon[31695]: pgmap v51: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 KiB/s wr, 578 op/s 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-03-11T19:24:46.715 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-11T19:24:46.716 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-11T19:24:46.717 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-11T19:24:46.718 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-03-11T19:24:46.719 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-03-11T19:24:46.720 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-11T19:24:46.773 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-03-11T19:24:46.774 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-03-11T19:24:46.775 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-03-11T19:24:46.776 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-11T19:24:46.777 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-03-11T19:24:46.778 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-03-11T19:24:46.779 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-11T19:24:46.867 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-03-11T19:24:46.868 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-11T19:24:46.869 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-03-11T19:24:46.870 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-11T19:24:46.871 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-03-11T19:24:46.872 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-11T19:24:46.873 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-11T19:24:46.976 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-11T19:24:46.977 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-11T19:24:46.978 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-11T19:24:46.979 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-03-11T19:24:46.980 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-11T19:24:46.981 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-03-11T19:24:47.034 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-11T19:24:47.034 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-03-11T19:24:47.034 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-11T19:24:47.034 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-03-11T19:24:47.034 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-11T19:24:47.035 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-11T19:24:47.036 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-11T19:24:47.037 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-11T19:24:47.038 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-03-11T19:24:47.039 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-03-11T19:24:47.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-03-11T19:24:47.091 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-11T19:24:47.092 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-11T19:24:47.093 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-03-11T19:24:47.094 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-03-11T19:24:47.095 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-11T19:24:47.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-03-11T19:24:47.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-11T19:24:47.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-03-11T19:24:47.190 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-03-11T19:24:47.191 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-03-11T19:24:47.192 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-11T19:24:47.193 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-03-11T19:24:47.194 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-11T19:24:47.195 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-11T19:24:47.287 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-03-11T19:24:47.288 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-03-11T19:24:47.289 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-11T19:24:47.290 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-03-11T19:24:47.291 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-11T19:24:47.292 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-11T19:24:47.334 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-11T19:24:47.335 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-03-11T19:24:47.336 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-11T19:24:47.337 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-11T19:24:47.338 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-11T19:24:47.339 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-03-11T19:24:48.086 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-11T19:24:48.086 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-03-11T19:24:48.087 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-03-11T19:24:48.088 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-03-11T19:24:48.089 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:48.090 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:48.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:48.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:48.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-11T19:24:48.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:48.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:48 smithi106 ceph-mon[35889]: pgmap v52: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 701 KiB/s wr, 603 op/s 2024-03-11T19:24:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:48 smithi072 ceph-mon[31695]: pgmap v52: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 701 KiB/s wr, 603 op/s 2024-03-11T19:24:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:48 smithi143 ceph-mon[36013]: pgmap v52: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 701 KiB/s wr, 603 op/s 2024-03-11T19:24:50.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:50 smithi106 ceph-mon[35889]: pgmap v53: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 695 KiB/s wr, 169 op/s 2024-03-11T19:24:50.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:50 smithi072 ceph-mon[31695]: pgmap v53: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 695 KiB/s wr, 169 op/s 2024-03-11T19:24:50.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:50 smithi143 ceph-mon[36013]: pgmap v53: 129 pgs: 129 active+clean; 9.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 695 KiB/s wr, 169 op/s 2024-03-11T19:24:51.273 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:51.274 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:51.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:51.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:51.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:51.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:51.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:52 smithi072 ceph-mon[31695]: pgmap v54: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 755 KiB/s wr, 316 op/s 2024-03-11T19:24:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:52 smithi143 ceph-mon[36013]: pgmap v54: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 755 KiB/s wr, 316 op/s 2024-03-11T19:24:52.881 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-11T19:24:52.881 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:52.881 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:52.881 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-11T19:24:52.882 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:52.883 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:52.884 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:52.885 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:52.886 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:52.887 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-11T19:24:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:52 smithi106 ceph-mon[35889]: pgmap v54: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 755 KiB/s wr, 316 op/s 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:53.507 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:53.508 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:53.509 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:53.510 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:53.511 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:53.512 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:53.513 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:54.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:54 smithi072 ceph-mon[31695]: pgmap v55: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 747 KiB/s wr, 171 op/s 2024-03-11T19:24:54.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:54 smithi143 ceph-mon[36013]: pgmap v55: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 747 KiB/s wr, 171 op/s 2024-03-11T19:24:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:54 smithi106 ceph-mon[35889]: pgmap v55: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 747 KiB/s wr, 171 op/s 2024-03-11T19:24:56.043 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:56.043 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:56.044 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-11T19:24:56.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:56.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-11T19:24:56.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:56.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:56.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-11T19:24:56.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:56.051 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-11T19:24:56.052 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:56.053 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:56.054 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:56.055 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:56.056 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-11T19:24:56.056 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:56.056 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:56.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:56 smithi072 ceph-mon[31695]: pgmap v56: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 748 KiB/s wr, 171 op/s 2024-03-11T19:24:56.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:56 smithi143 ceph-mon[36013]: pgmap v56: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 748 KiB/s wr, 171 op/s 2024-03-11T19:24:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:56 smithi106 ceph-mon[35889]: pgmap v56: 129 pgs: 129 active+clean; 174 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 748 KiB/s wr, 171 op/s 2024-03-11T19:24:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 -- 172.21.15.72:0/3043860810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 msgr2=0x7f0e40110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 --2- 172.21.15.72:0/3043860810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e40110260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009c10 tx=0x7f0e3802bef0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 -- 172.21.15.72:0/3043860810 shutdown_connections 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 --2- 172.21.15.72:0/3043860810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e40110260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 --2- 172.21.15.72:0/3043860810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e40103f70 0x7f0e40107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 --2- 172.21.15.72:0/3043860810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e401035c0 0x7f0e401039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 -- 172.21.15.72:0/3043860810 >> 172.21.15.72:0/3043860810 conn(0x7f0e400fee40 msgr2=0x7f0e40101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 -- 172.21.15.72:0/3043860810 shutdown_connections 2024-03-11T19:24:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.678+0000 7f0e46800640 1 -- 172.21.15.72:0/3043860810 wait complete. 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.679+0000 7f0e46800640 1 Processor -- start 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.679+0000 7f0e46800640 1 -- start start 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 0x7f0e4019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e4019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 0x7f0e40199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4019a200 con 0x7f0e40199730 2024-03-11T19:24:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4019a370 con 0x7f0e40108690 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e46800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4019a530 con 0x7f0e401035c0 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e4019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e4019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56538/0 (socket says 172.21.15.72:56538) 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e36a00640 1 -- 172.21.15.72:0/1037654087 learned_addr learned my addr 172.21.15.72:0/1037654087 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 0x7f0e4019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e3f400640 1 -- 172.21.15.72:0/1037654087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 msgr2=0x7f0e4019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e4019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 0x7f0e40199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.680+0000 7f0e3f400640 1 -- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 msgr2=0x7f0e40199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.681+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 0x7f0e40199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.681+0000 7f0e3f400640 1 -- 172.21.15.72:0/1037654087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38009840 con 0x7f0e401035c0 2024-03-11T19:24:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.681+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 0x7f0e40199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 0x7f0e4019f670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000e9f0 tx=0x7f0e3000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e3cc00640 1 -- 172.21.15.72:0/1037654087 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30009640 con 0x7f0e401035c0 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e3cc00640 1 -- 172.21.15.72:0/1037654087 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e3000ccc0 con 0x7f0e401035c0 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e3cc00640 1 -- 172.21.15.72:0/1037654087 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30018750 con 0x7f0e401035c0 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e401a64d0 con 0x7f0e401035c0 2024-03-11T19:24:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.682+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e401a6960 con 0x7f0e401035c0 2024-03-11T19:24:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.687+0000 7f0e49439740 1 Processor -- start 2024-03-11T19:24:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.687+0000 7f0e49439740 1 -- start start 2024-03-11T19:24:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f51f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f520230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f4db620 con 0x55914f51fdb0 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f48d8c0 con 0x55914f529330 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e49439740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f48adf0 con 0x55914f520b20 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56542/0 (socket says 172.21.15.72:56542) 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 -- 172.21.15.72:0/2072529125 learned_addr learned my addr 172.21.15.72:0/2072529125 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f51f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e3cc00640 1 -- 172.21.15.72:0/1037654087 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0e300109a0 con 0x7f0e401035c0 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 -- 172.21.15.72:0/2072529125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 msgr2=0x55914f51f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f51f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.688+0000 7f0e36a00640 1 -- 172.21.15.72:0/2072529125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 msgr2=0x55914f520230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e36a00640 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f520230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e36a00640 1 -- 172.21.15.72:0/2072529125 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55914f407b00 con 0x55914f529330 2024-03-11T19:24:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e36a00640 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009cb0 tx=0x7f0e28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215f329a5c6a140b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e34c00640 1 -- 172.21.15.72:0/2072529125 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28019630 con 0x55914f529330 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e34c00640 1 -- 172.21.15.72:0/2072529125 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e28009e60 con 0x55914f529330 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.689+0000 7f0e34c00640 1 -- 172.21.15.72:0/2072529125 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800ec40 con 0x55914f529330 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.690+0000 7f0e3cc00640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 0x7f0e240a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.690+0000 7f0e3cc00640 1 -- 172.21.15.72:0/1037654087 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f0e30014070 con 0x7f0e401035c0 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.690+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 0x7f0e240a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.690+0000 7f0e49439740 1 -- 172.21.15.72:0/2072529125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 msgr2=0x55914f52b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.690+0000 7f0e49439740 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009cb0 tx=0x7f0e28011810 comp rx=0 tx=0).stop 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 -- 172.21.15.72:0/2072529125 shutdown_connections 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f529330 0x55914f52b720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f520230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 --2- 172.21.15.72:0/2072529125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f51f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 -- 172.21.15.72:0/2072529125 >> 172.21.15.72:0/2072529125 conn(0x55914f4e74b0 msgr2=0x55914f51aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 -- 172.21.15.72:0/2072529125 shutdown_connections 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 -- 172.21.15.72:0/2072529125 wait complete. 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 0x7f0e240a4180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e38006010 tx=0x7f0e3802ce60 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.691+0000 7f0e49439740 1 -- start start 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 Processor -- start 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f520b20 0x55914f526940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 0x55914f526e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f4865d0 con 0x55914f529330 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f42bb50 con 0x55914f520b20 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f42b7f0 con 0x55914f51fdb0 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e49439740 10 client.0 _ll_register_callbacks cb 0x55914f43f550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:58264/0 (socket says 172.21.15.72:58264) 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e3f400640 1 -- 172.21.15.72:0/1547440990 learned_addr learned my addr 172.21.15.72:0/1547440990 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 0x55914f526e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.692+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f520b20 0x55914f526940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 -- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f520b20 msgr2=0x55914f526940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f520b20 0x55914f526940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 -- 172.21.15.72:0/1547440990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 msgr2=0x55914f526e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 0x55914f526e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 -- 172.21.15.72:0/1547440990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e28009910 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 0x55914f526e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e300cb350 tx=0x7f0e3009a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4662eeaef8bdda62 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300852b0 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f0e3009e8d0 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55914f458cd0 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.693+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e300a2f20 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 10 client.25606 fetch_fsmap learned FSMap version 27 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55914f484c50 con 0x55914f51fdb0 2024-03-11T19:24:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e300b2070 con 0x55914f51fdb0 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0e300964d0 con 0x55914f51fdb0 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 10 client.25606 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 20 client.25606 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f539410 0x55914f53b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 4 client.25606 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55914f47cce0 con 0x55914f539410 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.694+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f539410 0x55914f53b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.695+0000 7f0e2d600640 10 client.25606 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.695+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f539410 0x55914f53b800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628a55b2c5d8ca6b server_cookie=18d16923d5562b84 in_seq=0 out_seq=0 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e2d600640 1 -- 172.21.15.72:0/1547440990 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55914f47cce0 con 0x55914f539410 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e2d600640 10 client.25606 handle_command_reply: tid=0 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 client.25606 shutdown 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 msgr2=0x55914f526400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e300cb350 tx=0x7f0e3009a3b0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 shutdown_connections 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f539410 0x55914f53b800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f529330 0x55914f526e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f520b20 0x55914f526940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e2d600640 0 client.25606 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 --2- 172.21.15.72:0/1547440990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f526400 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 >> 172.21.15.72:0/1547440990 conn(0x55914f4e74b0 msgr2=0x55914f51c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 shutdown_connections 2024-03-11T19:24:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 1 -- 172.21.15.72:0/1547440990 wait complete. 2024-03-11T19:24:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.696+0000 7f0e49439740 20 client.25606 trim_cache size 0 max 16384 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.712+0000 7f0e49439740 1 Processor -- start 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 -- start start 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f528b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f520b20 0x55914f530690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f1cee60 con 0x55914f520b20 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f51baa0 con 0x55914f4e74b0 2024-03-11T19:24:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e49439740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f539210 con 0x55914f51fdb0 2024-03-11T19:24:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f520b20 0x55914f530690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f528b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:58276/0 (socket says 172.21.15.72:58276) 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.713+0000 7f0e36a00640 1 -- 172.21.15.72:0/3301786039 learned_addr learned my addr 172.21.15.72:0/3301786039 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 -- 172.21.15.72:0/3301786039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 msgr2=0x55914f528b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f528b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 -- 172.21.15.72:0/3301786039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f520b20 msgr2=0x55914f530690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f520b20 0x55914f530690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 -- 172.21.15.72:0/3301786039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55914f421530 con 0x55914f51fdb0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e3f400640 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f528b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e36a00640 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009cb0 tx=0x7f0e2800e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace0a10beb5a6ffd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e2fe00640 1 -- 172.21.15.72:0/3301786039 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28028ea0 con 0x55914f51fdb0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e2fe00640 1 -- 172.21.15.72:0/3301786039 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e28005050 con 0x55914f51fdb0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.714+0000 7f0e2fe00640 1 -- 172.21.15.72:0/3301786039 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280281e0 con 0x55914f51fdb0 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- 172.21.15.72:0/3301786039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 msgr2=0x55914f530150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009cb0 tx=0x7f0e2800e3b0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- 172.21.15.72:0/3301786039 shutdown_connections 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f520b20 0x55914f530690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f51fdb0 0x55914f530150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 --2- 172.21.15.72:0/3301786039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f528b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- 172.21.15.72:0/3301786039 >> 172.21.15.72:0/3301786039 conn(0x55914f52b900 msgr2=0x55914f524cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- 172.21.15.72:0/3301786039 shutdown_connections 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- 172.21.15.72:0/3301786039 wait complete. 2024-03-11T19:24:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.715+0000 7f0e49439740 1 -- start start 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 Processor -- start 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f5237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f58f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f58fdf0 con 0x55914f51fdb0 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f58ff30 con 0x55914f4e74b0 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55914f590070 con 0x55914f520b20 2024-03-11T19:24:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e49439740 10 client.0 _ll_register_callbacks cb 0x7f0e30009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e3f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f5237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35052/0 (socket says 172.21.15.72:35052) 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.716+0000 7f0e36a00640 1 -- 172.21.15.72:0/1706665736 learned_addr learned my addr 172.21.15.72:0/1706665736 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f58f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 -- 172.21.15.72:0/1706665736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 msgr2=0x55914f58f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f58f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 -- 172.21.15.72:0/1706665736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 msgr2=0x55914f523d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 -- 172.21.15.72:0/1706665736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55914f484c50 con 0x55914f4e74b0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3fe00640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f58f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e3f400640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f5237e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e3009a710 tx=0x7f0e3009a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ab1d5bc77fac3f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:24:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3009ae80 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55914f4abd30 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f0e3009e8d0 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.717+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e300a2f20 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 10 client.25622 fetch_fsmap learned FSMap version 27 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55914f407b00 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e3007d020 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0e30099cf0 con 0x55914f4e74b0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 10 client.25622 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 20 client.25622 populate_metadata read hostname 'smithi072' 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f529330 0x55914f5936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 4 client.25622 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55914f186600 con 0x55914f529330 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.718+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f529330 0x55914f5936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.719+0000 7f0e36a00640 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f529330 0x55914f5936d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44dfd9bd75223686 server_cookie=2bb5dfa234033af6 in_seq=0 out_seq=0 2024-03-11T19:24:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.719+0000 7f0e2d600640 10 client.25622 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e2d600640 1 -- 172.21.15.72:0/1706665736 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55914f186600 con 0x55914f529330 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e2d600640 10 client.25622 handle_command_reply: tid=0 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 client.25622 shutdown 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 msgr2=0x55914f5237e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f5237e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e3009a710 tx=0x7f0e3009a2c0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 shutdown_connections 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55914f529330 0x55914f5936d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55914f520b20 0x55914f58f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55914f51fdb0 0x55914f523d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e2d600640 0 client.25622 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 --2- 172.21.15.72:0/1706665736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914f4e74b0 0x55914f5237e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 >> 172.21.15.72:0/1706665736 conn(0x55914f52b900 msgr2=0x55914f526350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 shutdown_connections 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.726+0000 7f0e49439740 1 -- 172.21.15.72:0/1706665736 wait complete. 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e49439740 20 client.25622 trim_cache size 0 max 16384 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:24:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 msgr2=0x7f0e240a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 0x7f0e240a4180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e38006010 tx=0x7f0e3802ce60 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 msgr2=0x7f0e4019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 0x7f0e4019f670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000e9f0 tx=0x7f0e3000edf0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 shutdown_connections 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e240a1cc0 0x7f0e240a4180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e40199730 0x7f0e40199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e40108690 0x7f0e4019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 --2- 172.21.15.72:0/1037654087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e401035c0 0x7f0e4019f670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.727+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 >> 172.21.15.72:0/1037654087 conn(0x7f0e400fee40 msgr2=0x7f0e400ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.728+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 shutdown_connections 2024-03-11T19:24:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:24:57.728+0000 7f0e46800640 1 -- 172.21.15.72:0/1037654087 wait complete. 2024-03-11T19:24:58.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:24:58 smithi072 ceph-mon[31695]: pgmap v57: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 943 KiB/s wr, 234 op/s 2024-03-11T19:24:58.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:24:58 smithi143 ceph-mon[36013]: pgmap v57: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 943 KiB/s wr, 234 op/s 2024-03-11T19:24:58.847 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:58.847 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:58.848 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:58.849 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-11T19:24:58.850 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-11T19:24:58.851 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-11T19:24:58.852 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:58.853 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:58.853 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:58.853 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-11T19:24:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:24:58 smithi106 ceph-mon[35889]: pgmap v57: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 943 KiB/s wr, 234 op/s 2024-03-11T19:24:59.393 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:59.393 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:59.393 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:59.393 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:59.394 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:59.395 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:59.396 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:59.397 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-03-11T19:24:59.398 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-11T19:25:00.141 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-11T19:25:00.142 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-11T19:25:00.143 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-03-11T19:25:00.144 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-11T19:25:00.145 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-11T19:25:00.146 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-03-11T19:25:00.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:00 smithi072 ceph-mon[31695]: pgmap v58: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 255 KiB/s wr, 210 op/s 2024-03-11T19:25:00.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:00 smithi143 ceph-mon[36013]: pgmap v58: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 255 KiB/s wr, 210 op/s 2024-03-11T19:25:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:00 smithi106 ceph-mon[35889]: pgmap v58: 129 pgs: 129 active+clean; 128 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 255 KiB/s wr, 210 op/s 2024-03-11T19:25:02.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:02 smithi072 ceph-mon[31695]: pgmap v59: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 494 KiB/s wr, 297 op/s 2024-03-11T19:25:02.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:02 smithi143 ceph-mon[36013]: pgmap v59: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 494 KiB/s wr, 297 op/s 2024-03-11T19:25:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:02 smithi106 ceph-mon[35889]: pgmap v59: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 494 KiB/s wr, 297 op/s 2024-03-11T19:25:02.964 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-11T19:25:02.964 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-03-11T19:25:02.964 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-11T19:25:02.964 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-03-11T19:25:02.964 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-03-11T19:25:02.965 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-03-11T19:25:02.966 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-11T19:25:02.967 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-11T19:25:02.968 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-11T19:25:02.969 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-03-11T19:25:04.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:04 smithi143 ceph-mon[36013]: pgmap v60: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:04.868 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-11T19:25:04.868 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-03-11T19:25:04.868 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-11T19:25:04.868 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-11T19:25:04.869 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-03-11T19:25:04.870 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-11T19:25:04.871 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-03-11T19:25:04.872 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-11T19:25:04.873 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-03-11T19:25:04.874 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-11T19:25:04.874 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-03-11T19:25:04.874 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-11T19:25:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:04 smithi106 ceph-mon[35889]: pgmap v60: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:04 smithi072 ceph-mon[31695]: pgmap v60: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-11T19:25:05.501 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-03-11T19:25:05.502 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-11T19:25:05.503 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-03-11T19:25:05.504 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-11T19:25:05.505 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-11T19:25:05.506 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-03-11T19:25:06.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:06 smithi143 ceph-mon[36013]: pgmap v61: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:06 smithi106 ceph-mon[35889]: pgmap v61: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:06 smithi072 ceph-mon[31695]: pgmap v61: 129 pgs: 129 active+clean; 141 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 434 KiB/s wr, 149 op/s 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-11T19:25:07.091 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-03-11T19:25:07.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-03-11T19:25:07.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-03-11T19:25:07.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-03-11T19:25:07.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-03-11T19:25:07.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-11T19:25:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:08 smithi106 ceph-mon[35889]: pgmap v62: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 626 KiB/s wr, 216 op/s 2024-03-11T19:25:08.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:08 smithi072 ceph-mon[31695]: pgmap v62: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 626 KiB/s wr, 216 op/s 2024-03-11T19:25:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:08 smithi143 ceph-mon[36013]: pgmap v62: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 626 KiB/s wr, 216 op/s 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-11T19:25:09.929 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-03-11T19:25:09.930 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-11T19:25:09.931 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-03-11T19:25:09.932 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-03-11T19:25:09.933 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-11T19:25:09.934 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 -- 172.21.15.72:0/832274301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb618105190 msgr2=0x7fb61810cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 --2- 172.21.15.72:0/832274301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb618105190 0x7fb61810cd60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb610009be0 tx=0x7fb61002bef0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 -- 172.21.15.72:0/832274301 shutdown_connections 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 --2- 172.21.15.72:0/832274301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb618105190 0x7fb61810cd60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 --2- 172.21.15.72:0/832274301 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb618100a70 0x7fb618104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 --2- 172.21.15.72:0/832274301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6181000c0 0x7fb6181004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.874+0000 7fb617e00640 1 -- 172.21.15.72:0/832274301 >> 172.21.15.72:0/832274301 conn(0x7fb6180fb980 msgr2=0x7fb6180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.875+0000 7fb617e00640 1 -- 172.21.15.72:0/832274301 shutdown_connections 2024-03-11T19:25:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.875+0000 7fb617e00640 1 -- 172.21.15.72:0/832274301 wait complete. 2024-03-11T19:25:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.876+0000 7fb617e00640 1 Processor -- start 2024-03-11T19:25:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.876+0000 7fb617e00640 1 -- start start 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.876+0000 7fb617e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 0x7fb61819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6181995e0 0x7fb618199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618199fd0 con 0x7fb6181995e0 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61819a140 con 0x7fb618105190 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61819a300 con 0x7fb6181000c0 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:37478/0 (socket says 172.21.15.72:37478) 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 -- 172.21.15.72:0/1730222944 learned_addr learned my addr 172.21.15.72:0/1730222944 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616a00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 0x7fb61819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb617400640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6181995e0 0x7fb618199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 -- 172.21.15.72:0/1730222944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 msgr2=0x7fb61819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 0x7fb61819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 -- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6181995e0 msgr2=0x7fb618199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6181995e0 0x7fb618199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616000640 1 -- 172.21.15.72:0/1730222944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610009840 con 0x7fb618105190 2024-03-11T19:25:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.877+0000 7fb616a00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 0x7fb61819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:25:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb616000640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb60400e9c0 tx=0x7fb60400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb60f400640 1 -- 172.21.15.72:0/1730222944 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604009700 con 0x7fb618105190 2024-03-11T19:25:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb60f400640 1 -- 172.21.15.72:0/1730222944 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb604005500 con 0x7fb618105190 2024-03-11T19:25:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6181a62f0 con 0x7fb618105190 2024-03-11T19:25:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb60f400640 1 -- 172.21.15.72:0/1730222944 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604010430 con 0x7fb618105190 2024-03-11T19:25:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.878+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181a6850 con 0x7fb618105190 2024-03-11T19:25:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 Processor -- start 2024-03-11T19:25:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb60f400640 1 -- 172.21.15.72:0/1730222944 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb604005020 con 0x7fb618105190 2024-03-11T19:25:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 -- start start 2024-03-11T19:25:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8656ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b866b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b8612620 con 0x55b9b8657d60 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b85c48c0 con 0x55b9b8656ff0 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.882+0000 7fb620834740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b85c1df0 con 0x55b9b8668d80 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:37482/0 (socket says 172.21.15.72:37482) 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 -- 172.21.15.72:0/3750338638 learned_addr learned my addr 172.21.15.72:0/3750338638 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb617400640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8656ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616000640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b866b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 -- 172.21.15.72:0/3750338638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 msgr2=0x55b9b866b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b866b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 -- 172.21.15.72:0/3750338638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 msgr2=0x55b9b8656ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8656ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 -- 172.21.15.72:0/3750338638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9b853eb00 con 0x55b9b8656ff0 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb617400640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8656ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.883+0000 7fb616a00640 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb6080119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab907b5c86e6bb69 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb60f400640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 0x7fb5fc0a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb603e00640 1 -- 172.21.15.72:0/3750338638 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800f5c0 con 0x55b9b8656ff0 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb603e00640 1 -- 172.21.15.72:0/3750338638 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb60800fc00 con 0x55b9b8656ff0 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb603e00640 1 -- 172.21.15.72:0/3750338638 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608019d60 con 0x55b9b8656ff0 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb60f400640 1 -- 172.21.15.72:0/1730222944 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb604014070 con 0x7fb618105190 2024-03-11T19:25:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.884+0000 7fb617400640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 0x7fb5fc0a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- 172.21.15.72:0/3750338638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 msgr2=0x55b9b8657470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb6080119a0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb617400640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 0x7fb5fc0a40d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb61002e040 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- 172.21.15.72:0/3750338638 shutdown_connections 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b866b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8657470 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 --2- 172.21.15.72:0/3750338638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8656ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- 172.21.15.72:0/3750338638 >> 172.21.15.72:0/3750338638 conn(0x55b9b861e4b0 msgr2=0x55b9b8651ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- 172.21.15.72:0/3750338638 shutdown_connections 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- 172.21.15.72:0/3750338638 wait complete. 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.885+0000 7fb620834740 1 -- start start 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 Processor -- start 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8662c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b8668420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b85bd5d0 con 0x55b9b8657d60 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b8562b50 con 0x55b9b8656ff0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b85627f0 con 0x55b9b8668d80 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb620834740 10 client.0 _ll_register_callbacks cb 0x55b9b8576550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.886+0000 7fb616a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:37496/0 (socket says 172.21.15.72:37496) 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8662c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 -- 172.21.15.72:0/3472985353 learned_addr learned my addr 172.21.15.72:0/3472985353 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb617400640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b8668420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 -- 172.21.15.72:0/3472985353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 msgr2=0x55b9b8668420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b8668420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 -- 172.21.15.72:0/3472985353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 msgr2=0x55b9b8662c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8662c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 -- 172.21.15.72:0/3472985353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb608009d00 con 0x55b9b8656ff0 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616000640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8662c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:25:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb616a00640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb608005270 tx=0x7fb6080052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c45b640d87450c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.887+0000 7fb600c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608019e60 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb608019410 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9b858fcd0 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb608019610 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb620834740 10 client.25634 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b9b85bbc50 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.888+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9b858fcd0 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb60802b060 con 0x55b9b8656ff0 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 10 client.25634 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 20 client.25634 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86814b0 0x55b9b86838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 4 client.25634 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:25:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.889+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9b85b3ce0 con 0x55b9b86814b0 2024-03-11T19:25:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.890+0000 7fb616000640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86814b0 0x55b9b86838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:10 smithi106 ceph-mon[35889]: pgmap v63: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 430 KiB/s wr, 153 op/s 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.890+0000 7fb616000640 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86814b0 0x55b9b86838c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1600489683daba server_cookie=1df835f0ad5a4b0f in_seq=0 out_seq=0 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.890+0000 7fb600c00640 10 client.25634 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb600c00640 1 -- 172.21.15.72:0/3472985353 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9b85b3ce0 con 0x55b9b86814b0 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb600c00640 10 client.25634 handle_command_reply: tid=0 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 client.25634 shutdown 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 msgr2=0x55b9b8662700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb608005270 tx=0x7fb6080052a0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 shutdown_connections 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86814b0 0x55b9b86838c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b8668420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b8662c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 --2- 172.21.15.72:0/3472985353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8656ff0 0x55b9b8662700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb600c00640 0 client.25634 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 >> 172.21.15.72:0/3472985353 conn(0x55b9b861e4b0 msgr2=0x55b9b8653a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 shutdown_connections 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 1 -- 172.21.15.72:0/3472985353 wait complete. 2024-03-11T19:25:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.891+0000 7fb620834740 20 client.25634 trim_cache size 0 max 16384 2024-03-11T19:25:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 Processor -- start 2024-03-11T19:25:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 -- start start 2024-03-11T19:25:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8656ff0 0x55b9b86683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b865d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b8305e60 con 0x55b9b8656ff0 2024-03-11T19:25:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b865de10 con 0x55b9b8657d60 2024-03-11T19:25:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.908+0000 7fb620834740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b8688ad0 con 0x55b9b8668d80 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb617400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b865d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:37512/0 (socket says 172.21.15.72:37512) 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 -- 172.21.15.72:0/211665173 learned_addr learned my addr 172.21.15.72:0/211665173 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 -- 172.21.15.72:0/211665173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 msgr2=0x55b9b865d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b865d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 -- 172.21.15.72:0/211665173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8656ff0 msgr2=0x55b9b86683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8656ff0 0x55b9b86683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.909+0000 7fb616000640 1 -- 172.21.15.72:0/211665173 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9b8558530 con 0x55b9b8657d60 2024-03-11T19:25:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.910+0000 7fb616000640 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb604096af0 tx=0x7fb604085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7317ee9a843eb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.910+0000 7fb603400640 1 -- 172.21.15.72:0/211665173 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604005670 con 0x55b9b8657d60 2024-03-11T19:25:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.910+0000 7fb603400640 1 -- 172.21.15.72:0/211665173 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb60409ad90 con 0x55b9b8657d60 2024-03-11T19:25:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.910+0000 7fb603400640 1 -- 172.21.15.72:0/211665173 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040987c0 con 0x55b9b8657d60 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 -- 172.21.15.72:0/211665173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 msgr2=0x55b9b865d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb604096af0 tx=0x7fb604085040 comp rx=0 tx=0).stop 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 -- 172.21.15.72:0/211665173 shutdown_connections 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8668d80 0x55b9b865d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8657d60 0x55b9b865d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 --2- 172.21.15.72:0/211665173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8656ff0 0x55b9b86683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 -- 172.21.15.72:0/211665173 >> 172.21.15.72:0/211665173 conn(0x55b9b861e4b0 msgr2=0x55b9b8651810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.911+0000 7fb620834740 1 -- 172.21.15.72:0/211665173 shutdown_connections 2024-03-11T19:25:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.912+0000 7fb620834740 1 -- 172.21.15.72:0/211665173 wait complete. 2024-03-11T19:25:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.912+0000 7fb620834740 1 -- start start 2024-03-11T19:25:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.913+0000 7fb620834740 1 Processor -- start 2024-03-11T19:25:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.913+0000 7fb620834740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b86cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8668d80 0x55b9b86d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b86cc750 con 0x55b9b8657d60 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b86d6f00 con 0x55b9b8668d80 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9b86d7040 con 0x55b9b8656ff0 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb620834740 10 client.0 _ll_register_callbacks cb 0x7fb60801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:41034/0 (socket says 172.21.15.72:41034) 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb617400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8668d80 0x55b9b86d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b86cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 -- 172.21.15.72:0/1521493283 learned_addr learned my addr 172.21.15.72:0/1521493283 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 -- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8668d80 msgr2=0x55b9b86d69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8668d80 0x55b9b86d69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 -- 172.21.15.72:0/1521493283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 msgr2=0x55b9b86cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b86cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616a00640 1 -- 172.21.15.72:0/1521493283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181a6850 con 0x55b9b8656ff0 2024-03-11T19:25:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.914+0000 7fb616000640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b86cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:25:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb616a00640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6080059d0 tx=0x7fb608009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bce528fddfe767 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb600c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:25:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608019610 con 0x55b9b8656ff0 2024-03-11T19:25:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fb608000e40 con 0x55b9b8656ff0 2024-03-11T19:25:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9b85e2d30 con 0x55b9b8656ff0 2024-03-11T19:25:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb608019770 con 0x55b9b8656ff0 2024-03-11T19:25:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb620834740 10 client.25630 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.915+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b9b857e2c0 con 0x55b9b8656ff0 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b9b85e2d30 con 0x55b9b8656ff0 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb60802b040 con 0x55b9b8656ff0 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 10 client.25630 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 20 client.25630 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86d84f0 0x55b9b86da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 4 client.25630 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9b82bd600 con 0x55b9b86d84f0 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.916+0000 7fb616000640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86d84f0 0x55b9b86da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.917+0000 7fb600c00640 10 client.25630 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.917+0000 7fb616000640 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86d84f0 0x55b9b86da910 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14fbe5f7777e7738 server_cookie=e1eba57963ce79d9 in_seq=0 out_seq=0 2024-03-11T19:25:10.929 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:10 smithi072 ceph-mon[31695]: pgmap v63: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 430 KiB/s wr, 153 op/s 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.929+0000 7fb600c00640 1 -- 172.21.15.72:0/1521493283 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9b82bd600 con 0x55b9b86d84f0 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.929+0000 7fb600c00640 10 client.25630 handle_command_reply: tid=0 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.929+0000 7fb620834740 1 client.25630 shutdown 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 msgr2=0x55b9b86cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6080059d0 tx=0x7fb608009c90 comp rx=0 tx=0).stop 2024-03-11T19:25:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 shutdown_connections 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55b9b86d84f0 0x55b9b86da910 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b9b8668d80 0x55b9b86d69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b9b8657d60 0x55b9b86cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb600c00640 0 client.25630 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 --2- 172.21.15.72:0/1521493283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9b8656ff0 0x55b9b86cb940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 >> 172.21.15.72:0/1521493283 conn(0x55b9b861e4b0 msgr2=0x55b9b8651810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 shutdown_connections 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 1 -- 172.21.15.72:0/1521493283 wait complete. 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.930+0000 7fb620834740 20 client.25630 trim_cache size 0 max 16384 2024-03-11T19:25:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 msgr2=0x7fb5fc0a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 0x7fb5fc0a40d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb61002e040 comp rx=0 tx=0).stop 2024-03-11T19:25:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 msgr2=0x7fb61819f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb60400e9c0 tx=0x7fb60400edc0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 shutdown_connections 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb5fc0a1c10 0x7fb5fc0a40d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6181995e0 0x7fb618199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb618105190 0x7fb61819f970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 --2- 172.21.15.72:0/1730222944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6181000c0 0x7fb61819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 >> 172.21.15.72:0/1730222944 conn(0x7fb6180fb980 msgr2=0x7fb618069760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 shutdown_connections 2024-03-11T19:25:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:10.931+0000 7fb617e00640 1 -- 172.21.15.72:0/1730222944 wait complete. 2024-03-11T19:25:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:10 smithi143 ceph-mon[36013]: pgmap v63: 129 pgs: 129 active+clean; 99 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 430 KiB/s wr, 153 op/s 2024-03-11T19:25:11.366 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-11T19:25:11.366 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-03-11T19:25:11.367 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-03-11T19:25:11.368 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-11T19:25:11.369 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-11T19:25:11.370 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-03-11T19:25:11.371 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-11T19:25:11.372 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-03-11T19:25:12.069 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-11T19:25:12.069 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-03-11T19:25:12.070 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-11T19:25:12.071 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-03-11T19:25:12.072 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-11T19:25:12.073 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-11T19:25:12.074 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-03-11T19:25:12.075 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-11T19:25:12.075 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-03-11T19:25:12.075 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-11T19:25:12.075 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-03-11T19:25:12.075 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-11T19:25:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:12 smithi106 ceph-mon[35889]: pgmap v64: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 595 KiB/s wr, 211 op/s 2024-03-11T19:25:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:12 smithi072 ceph-mon[31695]: pgmap v64: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 595 KiB/s wr, 211 op/s 2024-03-11T19:25:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:12 smithi143 ceph-mon[36013]: pgmap v64: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 595 KiB/s wr, 211 op/s 2024-03-11T19:25:14.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-03-11T19:25:14.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-11T19:25:14.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-03-11T19:25:14.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-11T19:25:14.275 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-03-11T19:25:14.276 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-11T19:25:14.277 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-03-11T19:25:14.278 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-03-11T19:25:14.279 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-11T19:25:14.280 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-03-11T19:25:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:14 smithi106 ceph-mon[35889]: pgmap v65: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:14 smithi072 ceph-mon[31695]: pgmap v65: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:14 smithi143 ceph-mon[36013]: pgmap v65: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:16.741 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:16 smithi143 ceph-mon[36013]: pgmap v66: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:16 smithi106 ceph-mon[35889]: pgmap v66: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:16 smithi072 ceph-mon[31695]: pgmap v66: 129 pgs: 129 active+clean; 99 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 357 KiB/s wr, 125 op/s 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-03-11T19:25:17.211 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-11T19:25:17.212 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-03-11T19:25:17.213 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-11T19:25:17.214 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-03-11T19:25:17.215 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-11T19:25:17.216 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-03-11T19:25:17.217 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-11T19:25:18.045 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-11T19:25:18.046 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-03-11T19:25:18.047 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-03-11T19:25:18.048 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-03-11T19:25:18.049 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-11T19:25:18.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-03-11T19:25:18.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-11T19:25:18.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-03-11T19:25:18.050 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-11T19:25:18.677 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-03-11T19:25:18.677 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-11T19:25:18.677 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-03-11T19:25:18.677 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-03-11T19:25:18.678 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-11T19:25:18.679 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-11T19:25:18.680 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-03-11T19:25:18.681 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-11T19:25:18.682 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-03-11T19:25:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:18 smithi106 ceph-mon[35889]: pgmap v67: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 583 KiB/s wr, 195 op/s 2024-03-11T19:25:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:18 smithi072 ceph-mon[31695]: pgmap v67: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 583 KiB/s wr, 195 op/s 2024-03-11T19:25:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:18 smithi143 ceph-mon[36013]: pgmap v67: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 583 KiB/s wr, 195 op/s 2024-03-11T19:25:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:19 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:25:19.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:19 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:25:19.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:19 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:25:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:20 smithi106 ceph-mon[35889]: pgmap v68: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 391 KiB/s wr, 129 op/s 2024-03-11T19:25:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:20 smithi072 ceph-mon[31695]: pgmap v68: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 391 KiB/s wr, 129 op/s 2024-03-11T19:25:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:20 smithi143 ceph-mon[36013]: pgmap v68: 129 pgs: 129 active+clean; 97 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 391 KiB/s wr, 129 op/s 2024-03-11T19:25:20.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-11T19:25:20.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-11T19:25:20.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-03-11T19:25:20.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-11T19:25:20.990 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-03-11T19:25:20.991 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-03-11T19:25:20.992 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-11T19:25:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:21 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:25:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:21 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:25:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:21 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:25:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:21 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:25:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:21 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:25:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:21 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:25:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:21 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:25:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:21 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:25:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:21 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:25:22.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:22 smithi106 ceph-mon[35889]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 619 KiB/s wr, 200 op/s 2024-03-11T19:25:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:22 smithi072 ceph-mon[31695]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 619 KiB/s wr, 200 op/s 2024-03-11T19:25:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:22 smithi143 ceph-mon[36013]: pgmap v69: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 619 KiB/s wr, 200 op/s 2024-03-11T19:25:23.492 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-03-11T19:25:23.492 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-11T19:25:23.492 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-03-11T19:25:23.492 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-11T19:25:23.493 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-03-11T19:25:23.494 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-03-11T19:25:23.495 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-03-11T19:25:23.496 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-11T19:25:23.497 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-11T19:25:24.092 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-11T19:25:24.093 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-11T19:25:24.094 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-03-11T19:25:24.095 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-03-11T19:25:24.096 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-03-11T19:25:24.097 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 -- 172.21.15.72:0/3239384041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103db0 msgr2=0x7f84bc107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 --2- 172.21.15.72:0/3239384041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103db0 0x7f84bc107df0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009930 tx=0x7f84a802bc20 comp rx=0 tx=0).stop 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 -- 172.21.15.72:0/3239384041 shutdown_connections 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 --2- 172.21.15.72:0/3239384041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 --2- 172.21.15.72:0/3239384041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103db0 0x7f84bc107df0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 --2- 172.21.15.72:0/3239384041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc103400 0x7f84bc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 -- 172.21.15.72:0/3239384041 >> 172.21.15.72:0/3239384041 conn(0x7f84bc0fec60 msgr2=0x7f84bc101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 -- 172.21.15.72:0/3239384041 shutdown_connections 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.091+0000 7f84c0c00640 1 -- 172.21.15.72:0/3239384041 wait complete. 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.092+0000 7f84c0c00640 1 Processor -- start 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.092+0000 7f84c0c00640 1 -- start start 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 0x7f84bc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc1996e0 0x7f84bc199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19a1b0 con 0x7f84bc1996e0 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19a320 con 0x7f84bc103400 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84c0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19a4e0 con 0x7f84bc1084d0 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84baa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 0x7f84bc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84baa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36124/0 (socket says 172.21.15.72:36124) 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84baa00640 1 -- 172.21.15.72:0/1602962884 learned_addr learned my addr 172.21.15.72:0/1602962884 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.093+0000 7f84bbe00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc1996e0 0x7f84bc199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 -- 172.21.15.72:0/1602962884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 msgr2=0x7f84bc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 -- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc1996e0 msgr2=0x7f84bc199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc1996e0 0x7f84bc199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 -- 172.21.15.72:0/1602962884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac009490 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84bb400640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 0x7f84bc19f5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00e9f0 tx=0x7f84ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84b3e00640 1 -- 172.21.15.72:0/1602962884 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00cd80 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84b3e00640 1 -- 172.21.15.72:0/1602962884 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84ac0035f0 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84b3e00640 1 -- 172.21.15.72:0/1602962884 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac010690 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a8009560 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.094+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1a67d0 con 0x7f84bc103400 2024-03-11T19:25:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.097+0000 7f84b3e00640 1 -- 172.21.15.72:0/1602962884 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f84ac003af0 con 0x7f84bc103400 2024-03-11T19:25:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 Processor -- start 2024-03-11T19:25:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 -- start start 2024-03-11T19:25:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 0x55806d0c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 0x55806d0c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d07d620 con 0x55806d0c1ff0 2024-03-11T19:25:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d02f8c0 con 0x55806d0c2d60 2024-03-11T19:25:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84c5035740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d02cdf0 con 0x55806d0cb570 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.098+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 0x55806d0c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36134/0 (socket says 172.21.15.72:36134) 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 0x55806d0c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40274/0 (socket says 172.21.15.72:40274) 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 -- 172.21.15.72:0/3045626645 learned_addr learned my addr 172.21.15.72:0/3045626645 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84bb400640 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 0x55806d0c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 -- 172.21.15.72:0/3045626645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 msgr2=0x55806d0c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 0x55806d0c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 -- 172.21.15.72:0/3045626645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 msgr2=0x55806d0c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 0x55806d0c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 -- 172.21.15.72:0/3045626645 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806cfa9b00 con 0x55806d0cb570 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84bb400640 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 0x55806d0c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84b3e00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 0x7f84a00a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84b3e00640 1 -- 172.21.15.72:0/1602962884 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f84ac014070 con 0x7f84bc103400 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.099+0000 7f84baa00640 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84a802c160 tx=0x7f84a802c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5492ce4f9207045 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.100+0000 7f84a7e00640 1 -- 172.21.15.72:0/3045626645 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a802dea0 con 0x55806d0cb570 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.100+0000 7f84a7e00640 1 -- 172.21.15.72:0/3045626645 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84a8004880 con 0x55806d0cb570 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.100+0000 7f84a7e00640 1 -- 172.21.15.72:0/3045626645 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8005050 con 0x55806d0cb570 2024-03-11T19:25:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.100+0000 7f84bbe00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 0x7f84a00a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- 172.21.15.72:0/3045626645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 msgr2=0x55806d0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84a802c160 tx=0x7f84a802c670 comp rx=0 tx=0).stop 2024-03-11T19:25:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- 172.21.15.72:0/3045626645 shutdown_connections 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0cd960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c1ff0 0x55806d0c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 --2- 172.21.15.72:0/3045626645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c2d60 0x55806d0c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- 172.21.15.72:0/3045626645 >> 172.21.15.72:0/3045626645 conn(0x55806d0894b0 msgr2=0x55806d0bcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- 172.21.15.72:0/3045626645 shutdown_connections 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- 172.21.15.72:0/3045626645 wait complete. 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84bbe00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 0x7f84a00a40d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005fd0 tx=0x7f84b4009500 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.101+0000 7f84c5035740 1 -- start start 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 Processor -- start 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c1ff0 0x55806d0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0285d0 con 0x55806d0c2d60 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806cfcdb50 con 0x55806d0c1ff0 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.102+0000 7f84c5035740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806cfcd7f0 con 0x55806d0cb570 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84c5035740 10 client.0 _ll_register_callbacks cb 0x55806cfe1550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84baa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36138/0 (socket says 172.21.15.72:36138) 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 -- 172.21.15.72:0/2828935338 learned_addr learned my addr 172.21.15.72:0/2828935338 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bb400640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c1ff0 0x55806d0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 -- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c1ff0 msgr2=0x55806d0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c1ff0 0x55806d0c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 -- 172.21.15.72:0/2828935338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 msgr2=0x55806d0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.103+0000 7f84bbe00640 1 -- 172.21.15.72:0/2828935338 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1108b0 con 0x55806d0cb570 2024-03-11T19:25:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.104+0000 7f84bbe00640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ee0 tx=0x7f84b4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c479ccbd1efd3fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.104+0000 7f84a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.104+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4016070 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.104+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806cffacd0 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f84b4011990 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b4011b50 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 10 client.25642 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55806d026c50 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55806cffacd0 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f84b4028420 con 0x55806d0cb570 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 10 client.25642 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 20 client.25642 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d0db620 0x55806d0dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 4 client.25642 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.105+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55806d01ece0 con 0x55806d0db620 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.106+0000 7f84bb400640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d0db620 0x55806d0dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.106+0000 7f84bb400640 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d0db620 0x55806d0dda60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d902a84716ec9d6 server_cookie=fc12b24d6ea9079a in_seq=0 out_seq=0 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.106+0000 7f84a4c00640 10 client.25642 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84a4c00640 1 -- 172.21.15.72:0/2828935338 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55806d01ece0 con 0x55806d0db620 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84a4c00640 10 client.25642 handle_command_reply: tid=0 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 client.25642 shutdown 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 msgr2=0x55806d0c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ee0 tx=0x7f84b4004ab0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 shutdown_connections 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d0db620 0x55806d0dda60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0cb570 0x55806d0c9060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 --2- 172.21.15.72:0/2828935338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0c1ff0 0x55806d0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84a4c00640 0 client.25642 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 >> 172.21.15.72:0/2828935338 conn(0x55806d0894b0 msgr2=0x55806d0bea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 shutdown_connections 2024-03-11T19:25:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 1 -- 172.21.15.72:0/2828935338 wait complete. 2024-03-11T19:25:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.107+0000 7f84c5035740 20 client.25642 trim_cache size 0 max 16384 2024-03-11T19:25:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 Processor -- start 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 -- start start 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806cd70e60 con 0x55806d0c2d60 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0d7020 con 0x55806d0cb570 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84c5035740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0e2a70 con 0x55806d0c1ff0 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84baa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36146/0 (socket says 172.21.15.72:36146) 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84bb400640 1 -- 172.21.15.72:0/2634418446 learned_addr learned my addr 172.21.15.72:0/2634418446 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.124+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:40298/0 (socket says 172.21.15.72:40298) 2024-03-11T19:25:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 -- 172.21.15.72:0/2634418446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 msgr2=0x55806d0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 -- 172.21.15.72:0/2634418446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 msgr2=0x55806d0c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 -- 172.21.15.72:0/2634418446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806cfc3530 con 0x55806d0c2d60 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84bbe00640 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.125+0000 7f84baa00640 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c9070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005ec0 tx=0x7f84a8004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd4979735076b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.126+0000 7f84a7400640 1 -- 172.21.15.72:0/2634418446 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004a40 con 0x55806d0c2d60 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.126+0000 7f84a7400640 1 -- 172.21.15.72:0/2634418446 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84a802ce90 con 0x55806d0c2d60 2024-03-11T19:25:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.126+0000 7f84a7400640 1 -- 172.21.15.72:0/2634418446 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8005650 con 0x55806d0c2d60 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- 172.21.15.72:0/2634418446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 msgr2=0x55806d0c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c9070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84a8005ec0 tx=0x7f84a8004440 comp rx=0 tx=0).stop 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- 172.21.15.72:0/2634418446 shutdown_connections 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0c9070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 --2- 172.21.15.72:0/2634418446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- 172.21.15.72:0/2634418446 >> 172.21.15.72:0/2634418446 conn(0x55806d0894b0 msgr2=0x55806d0c6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- 172.21.15.72:0/2634418446 shutdown_connections 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- 172.21.15.72:0/2634418446 wait complete. 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.127+0000 7f84c5035740 1 -- start start 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 Processor -- start 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0d0f80 con 0x55806d0c2d60 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0d10f0 con 0x55806d0cb570 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d0d12b0 con 0x55806d0c1ff0 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84c5035740 10 client.0 _ll_register_callbacks cb 0x7f84b4009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:36154/0 (socket says 172.21.15.72:36154) 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84bb400640 1 -- 172.21.15.72:0/804684039 learned_addr learned my addr 172.21.15.72:0/804684039 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.128+0000 7f84baa00640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 -- 172.21.15.72:0/804684039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 msgr2=0x55806d0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 -- 172.21.15.72:0/804684039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 msgr2=0x55806d0d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0d2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 -- 172.21.15.72:0/804684039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806d026c50 con 0x55806d0cb570 2024-03-11T19:25:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bb400640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84bbe00640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0d08b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009480 tx=0x7f84b4028cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fea12b8413f9e49 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b401f840 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f84b4011a20 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.129+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806d04dd30 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b4011c40 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 10 client.25658 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55806cfa9b00 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55806d04dd30 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f84b4036450 con 0x55806d0cb570 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 10 client.25658 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 20 client.25658 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d132ec0 0x55806d1352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 4 client.25658 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:25:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55806cd28600 con 0x55806d132ec0 2024-03-11T19:25:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.130+0000 7f84bb400640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d132ec0 0x55806d1352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.131+0000 7f84bb400640 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d132ec0 0x55806d1352e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4abe9c5728f447 server_cookie=d375a933559bacc8 in_seq=0 out_seq=0 2024-03-11T19:25:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.131+0000 7f84a4c00640 10 client.25658 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84a4c00640 1 -- 172.21.15.72:0/804684039 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55806cd28600 con 0x55806d132ec0 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84a4c00640 10 client.25658 handle_command_reply: tid=0 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 client.25658 shutdown 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 msgr2=0x55806d0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0d08b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009480 tx=0x7f84b4028cf0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 shutdown_connections 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x55806d132ec0 0x55806d1352e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55806d0cb570 0x55806d0d08b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55806d0c2d60 0x55806d0d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84a4c00640 0 client.25658 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 --2- 172.21.15.72:0/804684039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55806d0c1ff0 0x55806d0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.138+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 >> 172.21.15.72:0/804684039 conn(0x55806d0894b0 msgr2=0x55806d0c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.139+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 shutdown_connections 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.139+0000 7f84c5035740 1 -- 172.21.15.72:0/804684039 wait complete. 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.139+0000 7f84c5035740 20 client.25658 trim_cache size 0 max 16384 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:25:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 msgr2=0x7f84a00a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 0x7f84a00a40d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84b4005fd0 tx=0x7f84b4009500 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 msgr2=0x7f84bc19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 0x7f84bc19f5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00e9f0 tx=0x7f84ac00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 shutdown_connections 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f84a00a1c10 0x7f84a00a40d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84bc1996e0 0x7f84bc199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc1084d0 0x7f84bc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 --2- 172.21.15.72:0/1602962884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84bc103400 0x7f84bc19f5d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 >> 172.21.15.72:0/1602962884 conn(0x7f84bc0fec60 msgr2=0x7f84bc0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 shutdown_connections 2024-03-11T19:25:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:24.140+0000 7f84c0c00640 1 -- 172.21.15.72:0/1602962884 wait complete. 2024-03-11T19:25:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:24 smithi106 ceph-mon[35889]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:24 smithi072 ceph-mon[31695]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:24 smithi143 ceph-mon[36013]: pgmap v70: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-03-11T19:25:25.584 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-03-11T19:25:25.585 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-03-11T19:25:25.586 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-03-11T19:25:25.587 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-03-11T19:25:25.588 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-03-11T19:25:25.589 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-03-11T19:25:25.590 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-11T19:25:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:26 smithi106 ceph-mon[35889]: pgmap v71: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:26 smithi072 ceph-mon[31695]: pgmap v71: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:26 smithi143 ceph-mon[36013]: pgmap v71: 129 pgs: 129 active+clean; 111 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 454 KiB/s wr, 141 op/s 2024-03-11T19:25:28.432 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-03-11T19:25:28.432 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-11T19:25:28.433 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-03-11T19:25:28.434 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-11T19:25:28.435 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-03-11T19:25:28.436 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-03-11T19:25:28.437 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-11T19:25:28.438 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-03-11T19:25:28.438 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-11T19:25:28.438 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-03-11T19:25:28.438 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-11T19:25:28.438 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-03-11T19:25:28.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:28 smithi106 ceph-mon[35889]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 666 KiB/s wr, 196 op/s 2024-03-11T19:25:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:28 smithi072 ceph-mon[31695]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 666 KiB/s wr, 196 op/s 2024-03-11T19:25:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:28 smithi143 ceph-mon[36013]: pgmap v72: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 666 KiB/s wr, 196 op/s 2024-03-11T19:25:30.325 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-11T19:25:30.325 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-03-11T19:25:30.325 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-03-11T19:25:30.326 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-03-11T19:25:30.327 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-11T19:25:30.328 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-11T19:25:30.329 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-11T19:25:30.330 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-11T19:25:30.331 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-03-11T19:25:30.332 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-11T19:25:30.332 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-03-11T19:25:30.332 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-11T19:25:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:30 smithi106 ceph-mon[35889]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 439 KiB/s wr, 125 op/s 2024-03-11T19:25:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:30 smithi072 ceph-mon[31695]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 439 KiB/s wr, 125 op/s 2024-03-11T19:25:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:30 smithi143 ceph-mon[36013]: pgmap v73: 129 pgs: 129 active+clean; 309 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 439 KiB/s wr, 125 op/s 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-11T19:25:30.970 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-11T19:25:30.971 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-03-11T19:25:30.972 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-11T19:25:30.973 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-11T19:25:30.974 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-11T19:25:30.975 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-03-11T19:25:32.737 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-11T19:25:32.737 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-03-11T19:25:32.737 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-03-11T19:25:32.738 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-03-11T19:25:32.739 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-03-11T19:25:32.740 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-03-11T19:25:32.741 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-11T19:25:32.742 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-03-11T19:25:32.742 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-11T19:25:32.742 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-03-11T19:25:32.742 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-03-11T19:25:32.743 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-03-11T19:25:32.744 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-11T19:25:32.745 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-03-11T19:25:32.746 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-11T19:25:32.746 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-03-11T19:25:32.746 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-11T19:25:32.746 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-11T19:25:32.747 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-11T19:25:32.748 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-11T19:25:32.749 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-03-11T19:25:32.750 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-11T19:25:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:32 smithi106 ceph-mon[35889]: pgmap v74: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 588 KiB/s wr, 182 op/s 2024-03-11T19:25:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:32 smithi072 ceph-mon[31695]: pgmap v74: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 588 KiB/s wr, 182 op/s 2024-03-11T19:25:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:32 smithi143 ceph-mon[36013]: pgmap v74: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 588 KiB/s wr, 182 op/s 2024-03-11T19:25:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:34 smithi106 ceph-mon[35889]: pgmap v75: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:34 smithi072 ceph-mon[31695]: pgmap v75: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:34 smithi143 ceph-mon[36013]: pgmap v75: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-11T19:25:35.838 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-11T19:25:35.839 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-11T19:25:35.840 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-11T19:25:35.841 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-03-11T19:25:35.842 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-03-11T19:25:35.843 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-11T19:25:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:36 smithi106 ceph-mon[35889]: pgmap v76: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:36 smithi072 ceph-mon[31695]: pgmap v76: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:36 smithi143 ceph-mon[36013]: pgmap v76: 129 pgs: 129 active+clean; 274 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 361 KiB/s wr, 112 op/s 2024-03-11T19:25:36.983 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-11T19:25:36.984 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-03-11T19:25:36.985 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-11T19:25:36.986 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-11T19:25:36.987 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-03-11T19:25:36.988 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-11T19:25:36.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-03-11T19:25:36.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-11T19:25:36.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-03-11T19:25:36.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-11T19:25:36.989 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 -- 172.21.15.72:0/2325472602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073530 msgr2=0x7f7abc073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 --2- 172.21.15.72:0/2325472602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073530 0x7f7abc073910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009ca0 tx=0x7f7ab402c180 comp rx=0 tx=0).stop 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 -- 172.21.15.72:0/2325472602 shutdown_connections 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 --2- 172.21.15.72:0/2325472602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc10d840 0x7f7abc10fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 --2- 172.21.15.72:0/2325472602 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc073e50 0x7f7abc10d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 --2- 172.21.15.72:0/2325472602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073530 0x7f7abc073910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 -- 172.21.15.72:0/2325472602 >> 172.21.15.72:0/2325472602 conn(0x7f7abc0fca90 msgr2=0x7f7abc0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.296+0000 7f7ac1600640 1 -- 172.21.15.72:0/2325472602 shutdown_connections 2024-03-11T19:25:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.297+0000 7f7ac1600640 1 -- 172.21.15.72:0/2325472602 wait complete. 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 Processor -- start 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 -- start start 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073e50 0x7f7abc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc10d840 0x7f7abc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc199e10 con 0x7f7abc10d840 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc199f80 con 0x7f7abc073e50 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19a140 con 0x7f7abc073530 2024-03-11T19:25:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:37932/0 (socket says 172.21.15.72:37932) 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc10d840 0x7f7abc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.298+0000 7f7abbe00640 1 -- 172.21.15.72:0/812336647 learned_addr learned my addr 172.21.15.72:0/812336647 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abb400640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073e50 0x7f7abc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 -- 172.21.15.72:0/812336647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073e50 msgr2=0x7f7abc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073e50 0x7f7abc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 -- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc10d840 msgr2=0x7f7abc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc10d840 0x7f7abc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 -- 172.21.15.72:0/812336647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009900 con 0x7f7abc073530 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.299+0000 7f7abbe00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4005e30 tx=0x7f7ab4034830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.300+0000 7f7ab8c00640 1 -- 172.21.15.72:0/812336647 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4003e40 con 0x7f7abc073530 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.300+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc19a3c0 con 0x7f7abc073530 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.300+0000 7f7ab8c00640 1 -- 172.21.15.72:0/812336647 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ab4032c70 con 0x7f7abc073530 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.300+0000 7f7ab8c00640 1 -- 172.21.15.72:0/812336647 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4040450 con 0x7f7abc073530 2024-03-11T19:25:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.300+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a6540 con 0x7f7abc073530 2024-03-11T19:25:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.303+0000 7f7ab8c00640 1 -- 172.21.15.72:0/812336647 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7ab400a040 con 0x7f7abc073530 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.303+0000 7f7ac5830740 1 Processor -- start 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.303+0000 7f7ac5830740 1 -- start start 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x56019654a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196555960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x560196505620 con 0x560196549ff0 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601964b78c0 con 0x560196553570 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac5830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601964b4df0 con 0x56019654ad60 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x56019654a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:37940/0 (socket says 172.21.15.72:37940) 2024-03-11T19:25:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 -- 172.21.15.72:0/4217149906 learned_addr learned my addr 172.21.15.72:0/4217149906 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7abb400640 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196555960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 -- 172.21.15.72:0/4217149906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 msgr2=0x560196555960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196555960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 -- 172.21.15.72:0/4217149906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 msgr2=0x56019654a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.304+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x56019654a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ac0c00640 1 -- 172.21.15.72:0/4217149906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560196431b00 con 0x56019654ad60 2024-03-11T19:25:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ab8c00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 0x7f7aa00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c20 tx=0x7f7aac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b84ef90d3048b0a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ab8c00640 1 -- 172.21.15.72:0/812336647 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f7ab40f0340 con 0x7f7abc073530 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ab1200640 1 -- 172.21.15.72:0/4217149906 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac019630 con 0x56019654ad60 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ab1200640 1 -- 172.21.15.72:0/4217149906 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aac009e90 con 0x56019654ad60 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7ab1200640 1 -- 172.21.15.72:0/4217149906 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00eb00 con 0x56019654ad60 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.305+0000 7f7abbe00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 0x7f7aa00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 -- 172.21.15.72:0/4217149906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 msgr2=0x560196549ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7abbe00640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 0x7f7aa00a4050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40ed8d0 tx=0x7f7ab40f3e50 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c20 tx=0x7f7aac011810 comp rx=0 tx=0).stop 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 -- 172.21.15.72:0/4217149906 shutdown_connections 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196555960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x56019654a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 --2- 172.21.15.72:0/4217149906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x560196549ab0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 -- 172.21.15.72:0/4217149906 >> 172.21.15.72:0/4217149906 conn(0x5601965114b0 msgr2=0x560196544ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.306+0000 7f7ac5830740 1 -- 172.21.15.72:0/4217149906 shutdown_connections 2024-03-11T19:25:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.307+0000 7f7ac5830740 1 -- 172.21.15.72:0/4217149906 wait complete. 2024-03-11T19:25:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.307+0000 7f7ac5830740 1 -- start start 2024-03-11T19:25:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.307+0000 7f7ac5830740 1 Processor -- start 2024-03-11T19:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 0x5601965505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 0x560196550b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196553570 0x560196551040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601964b05d0 con 0x560196553570 2024-03-11T19:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560196455b50 con 0x56019654ad60 2024-03-11T19:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601964557f0 con 0x560196549ff0 2024-03-11T19:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac5830740 10 client.0 _ll_register_callbacks cb 0x560196469550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 0x5601965505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7abb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 0x560196550b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 0x5601965505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:37946/0 (socket says 172.21.15.72:37946) 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac0c00640 1 -- 172.21.15.72:0/431790512 learned_addr learned my addr 172.21.15.72:0/431790512 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.308+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196553570 0x560196551040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 -- 172.21.15.72:0/431790512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 msgr2=0x5601965505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 0x5601965505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 -- 172.21.15.72:0/431790512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196553570 msgr2=0x560196551040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196553570 0x560196551040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 -- 172.21.15.72:0/431790512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac0098d0 con 0x56019654ad60 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7abb400640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 0x560196550b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56019654aa90 tx=0x7f7aa800bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9171454d0173136 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8016070 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f7aa8003d90 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560196482cd0 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa8005020 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7ac5830740 10 client.25670 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.309+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601964aec50 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560196482cd0 con 0x56019654ad60 2024-03-11T19:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7aa801b470 con 0x56019654ad60 2024-03-11T19:25:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 10 client.25670 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 20 client.25670 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965636e0 0x560196565b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 4 client.25670 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601964a6ce0 con 0x5601965636e0 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.310+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965636e0 0x560196565b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965636e0 0x560196565b00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310f166ce6d6f821 server_cookie=53999591c1f04e8e in_seq=0 out_seq=0 2024-03-11T19:25:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7aa5600640 10 client.25670 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7aa5600640 1 -- 172.21.15.72:0/431790512 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601964a6ce0 con 0x5601965636e0 2024-03-11T19:25:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7aa5600640 10 client.25670 handle_command_reply: tid=0 2024-03-11T19:25:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7ac5830740 1 client.25670 shutdown 2024-03-11T19:25:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 msgr2=0x560196550b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.311+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 0x560196550b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56019654aa90 tx=0x7f7aa800bf30 comp rx=0 tx=0).stop 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 shutdown_connections 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965636e0 0x560196565b00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196553570 0x560196551040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56019654ad60 0x560196550b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 --2- 172.21.15.72:0/431790512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196549ff0 0x5601965505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7aa5600640 0 client.25670 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 >> 172.21.15.72:0/431790512 conn(0x5601965114b0 msgr2=0x560196546a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 shutdown_connections 2024-03-11T19:25:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 1 -- 172.21.15.72:0/431790512 wait complete. 2024-03-11T19:25:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.312+0000 7f7ac5830740 20 client.25670 trim_cache size 0 max 16384 2024-03-11T19:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.328+0000 7f7ac5830740 1 Processor -- start 2024-03-11T19:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.328+0000 7f7ac5830740 1 -- start start 2024-03-11T19:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.328+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.328+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56019654ad60 0x56019654ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196553570 0x56019654f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac5830740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601961f8e60 con 0x56019654ad60 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac5830740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601965600c0 con 0x560196549ff0 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac5830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560196561e00 con 0x560196553570 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7abb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56019654ad60 0x56019654ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:47006/0 (socket says 172.21.15.72:47006) 2024-03-11T19:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 -- 172.21.15.72:0/2973116252 learned_addr learned my addr 172.21.15.72:0/2973116252 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196553570 0x56019654f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 -- 172.21.15.72:0/2973116252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196553570 msgr2=0x56019654f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196553570 0x56019654f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 -- 172.21.15.72:0/2973116252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56019654ad60 msgr2=0x56019654ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56019654ad60 0x56019654ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.329+0000 7f7ac0c00640 1 -- 172.21.15.72:0/2973116252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019644b530 con 0x560196549ff0 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.330+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098a0 tx=0x7f7aac000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbce68986933136a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.330+0000 7f7aa7e00640 1 -- 172.21.15.72:0/2973116252 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02dd00 con 0x560196549ff0 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.330+0000 7f7aa7e00640 1 -- 172.21.15.72:0/2973116252 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aac02d050 con 0x560196549ff0 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.330+0000 7f7aa7e00640 1 -- 172.21.15.72:0/2973116252 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02d1c0 con 0x560196549ff0 2024-03-11T19:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- 172.21.15.72:0/2973116252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 msgr2=0x560196552cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098a0 tx=0x7f7aac000bc0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- 172.21.15.72:0/2973116252 shutdown_connections 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560196553570 0x56019654f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56019654ad60 0x56019654ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2973116252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196549ff0 0x560196552cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- 172.21.15.72:0/2973116252 >> 172.21.15.72:0/2973116252 conn(0x5601965114b0 msgr2=0x560196544810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- 172.21.15.72:0/2973116252 shutdown_connections 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- 172.21.15.72:0/2973116252 wait complete. 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.331+0000 7f7ac5830740 1 -- start start 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 Processor -- start 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x560196558eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196559930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601965b9bf0 con 0x560196549ff0 2024-03-11T19:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601965b9d30 con 0x560196553570 2024-03-11T19:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601965b9ea0 con 0x56019654ad60 2024-03-11T19:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.332+0000 7f7ac5830740 10 client.0 _ll_register_callbacks cb 0x7f7aa8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:37956/0 (socket says 172.21.15.72:37956) 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 -- 172.21.15.72:0/2109081646 learned_addr learned my addr 172.21.15.72:0/2109081646 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x560196558eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196559930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 -- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 msgr2=0x560196559930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196559930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 -- 172.21.15.72:0/2109081646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 msgr2=0x560196558eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x560196558eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7abb400640 1 -- 172.21.15.72:0/2109081646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560196431b00 con 0x56019654ad60 2024-03-11T19:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.333+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196559930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7abb400640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8003a90 tx=0x7f7aa80042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97a07170eec2da server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:25:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8016040 con 0x56019654ad60 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601964d5d30 con 0x56019654ad60 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9422+0+0 (secure 0 0 0) 0x7f7aa8000e40 con 0x56019654ad60 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa8003ed0 con 0x56019654ad60 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7ac5830740 10 client.25666 fetch_fsmap learned FSMap version 27 2024-03-11T19:25:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.334+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5601964712c0 con 0x56019654ad60 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5601964d5d30 con 0x56019654ad60 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 27) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7aa8031af0 con 0x56019654ad60 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 10 client.25666 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 20 client.25666 populate_metadata read hostname 'smithi072' 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965bb760 0x5601965bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:25:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 4 client.25666 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 --> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601961b0600 con 0x5601965bb760 2024-03-11T19:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.335+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965bb760 0x5601965bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.336+0000 7f7ac0c00640 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965bb760 0x5601965bdb80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92134cd01a1fbf0 server_cookie=79a66117cdc91df7 in_seq=0 out_seq=0 2024-03-11T19:25:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.336+0000 7f7aa5600640 10 client.25666 ms_handle_connect on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.342+0000 7f7aa5600640 1 -- 172.21.15.72:0/2109081646 <== mds.0 v2:172.21.15.106:6838/806155003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601961b0600 con 0x5601965bb760 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7aa5600640 10 client.25666 handle_command_reply: tid=0 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 client.25666 shutdown 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 msgr2=0x5601965593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8003a90 tx=0x7f7aa80042b0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 shutdown_connections 2024-03-11T19:25:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:6838/806155003,v1:172.21.15.106:6839/806155003] conn(0x5601965bb760 0x5601965bdb80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560196553570 0x560196559930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56019654ad60 0x5601965593f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 --2- 172.21.15.72:0/2109081646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560196549ff0 0x560196558eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 >> 172.21.15.72:0/2109081646 conn(0x5601965114b0 msgr2=0x56019654ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7aa5600640 0 client.25666 ms_handle_reset on v2:172.21.15.106:6838/806155003 2024-03-11T19:25:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 shutdown_connections 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 1 -- 172.21.15.72:0/2109081646 wait complete. 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.343+0000 7f7ac5830740 20 client.25666 trim_cache size 0 max 16384 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:{ 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "message": "", 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout: "return_code": 0 2024-03-11T19:25:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stdout:} 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 msgr2=0x7f7aa00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 0x7f7aa00a4050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40ed8d0 tx=0x7f7ab40f3e50 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 msgr2=0x7f7abc19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4005e30 tx=0x7f7ab4034830 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 shutdown_connections 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f7aa00a1b90 0x7f7aa00a4050 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7abc10d840 0x7f7abc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc073e50 0x7f7abc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 --2- 172.21.15.72:0/812336647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7abc073530 0x7f7abc19f5c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.344+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 >> 172.21.15.72:0/812336647 conn(0x7f7abc0fca90 msgr2=0x7f7abc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:25:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.345+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 shutdown_connections 2024-03-11T19:25:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi072.front.sepia.ceph.com.smithi072.stderr:2024-03-11T19:25:37.345+0000 7f7ac1600640 1 -- 172.21.15.72:0/812336647 wait complete. 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-03-11T19:25:37.566 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-03-11T19:25:37.567 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-03-11T19:25:37.568 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-03-11T19:25:37.569 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-11T19:25:37.570 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-03-11T19:25:37.571 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-03-11T19:25:37.674 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-03-11T19:25:37.675 INFO:tasks.workunit.client.0.smithi072.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-11T19:25:37.676 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:25:37.676 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:25:37.820 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-03-11T19:25:37.820 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-11T19:25:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:38 smithi106 ceph-mon[35889]: pgmap v77: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 554 KiB/s wr, 174 op/s 2024-03-11T19:25:38.909 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:38 smithi072 ceph-mon[31695]: pgmap v77: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 554 KiB/s wr, 174 op/s 2024-03-11T19:25:38.913 DEBUG:teuthology.parallel:result is None 2024-03-11T19:25:38.914 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T19:25:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:38 smithi143 ceph-mon[36013]: pgmap v77: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 554 KiB/s wr, 174 op/s 2024-03-11T19:25:39.026 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T19:25:39.027 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-11T19:25:39.040 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:25:39.058 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-11T19:25:39.068 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-11T19:25:39.068 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:25:39.068 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:25:39.068 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-11T19:25:39.201 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-11T19:25:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:40 smithi106 ceph-mon[35889]: pgmap v78: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 342 KiB/s wr, 120 op/s 2024-03-11T19:25:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:40 smithi072 ceph-mon[31695]: pgmap v78: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 342 KiB/s wr, 120 op/s 2024-03-11T19:25:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:40 smithi143 ceph-mon[36013]: pgmap v78: 129 pgs: 129 active+clean; 243 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 342 KiB/s wr, 120 op/s 2024-03-11T19:25:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:41 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:25:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:41 smithi106 ceph-mon[35889]: pgmap v79: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 504 KiB/s wr, 171 op/s 2024-03-11T19:25:41.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:41 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:25:41.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:41 smithi072 ceph-mon[31695]: pgmap v79: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 504 KiB/s wr, 171 op/s 2024-03-11T19:25:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:41 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:25:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:41 smithi143 ceph-mon[36013]: pgmap v79: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 504 KiB/s wr, 171 op/s 2024-03-11T19:25:44.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:44 smithi072 ceph-mon[31695]: pgmap v80: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:44.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:44 smithi143 ceph-mon[36013]: pgmap v80: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:44 smithi106 ceph-mon[35889]: pgmap v80: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:46.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:46 smithi072 ceph-mon[31695]: pgmap v81: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:46.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:46 smithi143 ceph-mon[36013]: pgmap v81: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:46 smithi106 ceph-mon[35889]: pgmap v81: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 356 KiB/s wr, 113 op/s 2024-03-11T19:25:48.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:48 smithi072 ceph-mon[31695]: pgmap v82: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 362 KiB/s wr, 125 op/s 2024-03-11T19:25:48.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:48 smithi143 ceph-mon[36013]: pgmap v82: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 362 KiB/s wr, 125 op/s 2024-03-11T19:25:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:48 smithi106 ceph-mon[35889]: pgmap v82: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 362 KiB/s wr, 125 op/s 2024-03-11T19:25:50.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:50 smithi072 ceph-mon[31695]: pgmap v83: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 169 KiB/s wr, 63 op/s 2024-03-11T19:25:50.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:50 smithi143 ceph-mon[36013]: pgmap v83: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 169 KiB/s wr, 63 op/s 2024-03-11T19:25:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:50 smithi106 ceph-mon[35889]: pgmap v83: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 169 KiB/s wr, 63 op/s 2024-03-11T19:25:52.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:52 smithi072 ceph-mon[31695]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 170 KiB/s wr, 65 op/s 2024-03-11T19:25:52.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:52 smithi143 ceph-mon[36013]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 170 KiB/s wr, 65 op/s 2024-03-11T19:25:52.697 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:52 smithi106 ceph-mon[35889]: pgmap v84: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 170 KiB/s wr, 65 op/s 2024-03-11T19:25:54.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:54 smithi143 ceph-mon[36013]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:54 smithi106 ceph-mon[35889]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:54 smithi072 ceph-mon[31695]: pgmap v85: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:56 smithi106 ceph-mon[35889]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:56 smithi072 ceph-mon[31695]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:56.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:56 smithi143 ceph-mon[36013]: pgmap v86: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-11T19:25:58.733 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:25:58 smithi072 ceph-mon[31695]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-11T19:25:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:25:58 smithi106 ceph-mon[35889]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-11T19:25:58.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:25:58 smithi143 ceph-mon[36013]: pgmap v87: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-11T19:26:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:00 smithi106 ceph-mon[35889]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-11T19:26:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:00 smithi072 ceph-mon[31695]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-11T19:26:00.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:00 smithi143 ceph-mon[36013]: pgmap v88: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-11T19:26:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:02 smithi106 ceph-mon[35889]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-03-11T19:26:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:02 smithi072 ceph-mon[31695]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-03-11T19:26:02.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:02 smithi143 ceph-mon[36013]: pgmap v89: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-03-11T19:26:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:04 smithi106 ceph-mon[35889]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:26:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:04 smithi072 ceph-mon[31695]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:26:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:04 smithi143 ceph-mon[36013]: pgmap v90: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:26:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:06 smithi106 ceph-mon[35889]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-03-11T19:26:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:06 smithi072 ceph-mon[31695]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-03-11T19:26:06.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:06 smithi143 ceph-mon[36013]: pgmap v91: 129 pgs: 129 active+clean; 20 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-03-11T19:26:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:08 smithi106 ceph-mon[35889]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:08 smithi072 ceph-mon[31695]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:08 smithi143 ceph-mon[36013]: pgmap v92: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:10 smithi106 ceph-mon[35889]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:26:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:10 smithi072 ceph-mon[31695]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:26:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:10 smithi143 ceph-mon[36013]: pgmap v93: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:26:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:12 smithi106 ceph-mon[35889]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:12 smithi072 ceph-mon[31695]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:12 smithi143 ceph-mon[36013]: pgmap v94: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:26:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:14 smithi106 ceph-mon[35889]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-11T19:26:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:14 smithi072 ceph-mon[31695]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-11T19:26:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:14 smithi143 ceph-mon[36013]: pgmap v95: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-11T19:26:16.742 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:16 smithi143 ceph-mon[36013]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 14 op/s 2024-03-11T19:26:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:16 smithi106 ceph-mon[35889]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 14 op/s 2024-03-11T19:26:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:16 smithi072 ceph-mon[31695]: pgmap v96: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 14 op/s 2024-03-11T19:26:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:18 smithi106 ceph-mon[35889]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-11T19:26:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:18 smithi072 ceph-mon[31695]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-11T19:26:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:18 smithi143 ceph-mon[36013]: pgmap v97: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-11T19:26:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:20 smithi106 ceph-mon[35889]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:26:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:20 smithi072 ceph-mon[31695]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:26:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:20 smithi143 ceph-mon[36013]: pgmap v98: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:26:21.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:21 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:26:21.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:21 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:26:21.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:21 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:26:22.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:22 smithi106 ceph-mon[35889]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:26:22.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:22 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:22.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:22 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:26:22.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:22 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:26:22.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:22 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:26:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:22 smithi072 ceph-mon[31695]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:26:22.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:22 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:22.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:22 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:26:22.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:22 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:26:22.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:22 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:26:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:22 smithi143 ceph-mon[36013]: pgmap v99: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:26:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:22 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:22 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:26:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:22 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:26:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:22 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:26:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:24 smithi106 ceph-mon[35889]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:24 smithi072 ceph-mon[31695]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:24 smithi143 ceph-mon[36013]: pgmap v100: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:26 smithi106 ceph-mon[35889]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:26:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:26 smithi072 ceph-mon[31695]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:26:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:26 smithi143 ceph-mon[36013]: pgmap v101: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:26:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:28 smithi106 ceph-mon[35889]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:28 smithi072 ceph-mon[31695]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:28 smithi143 ceph-mon[36013]: pgmap v102: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:30 smithi106 ceph-mon[35889]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:30 smithi072 ceph-mon[31695]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:30 smithi143 ceph-mon[36013]: pgmap v103: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:32 smithi106 ceph-mon[35889]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:32 smithi072 ceph-mon[31695]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:32 smithi143 ceph-mon[36013]: pgmap v104: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:34 smithi106 ceph-mon[35889]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:34 smithi072 ceph-mon[31695]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:34 smithi143 ceph-mon[36013]: pgmap v105: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:36 smithi106 ceph-mon[35889]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:36 smithi072 ceph-mon[31695]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:36 smithi143 ceph-mon[36013]: pgmap v106: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:37.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:37 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:37 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:37 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:26:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:38 smithi106 ceph-mon[35889]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:38 smithi072 ceph-mon[31695]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:38 smithi143 ceph-mon[36013]: pgmap v107: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:26:39.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:39 smithi106 ceph-mon[35889]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:39.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:39 smithi072 ceph-mon[31695]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:39.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:39 smithi143 ceph-mon[36013]: pgmap v108: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:42.689 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:42 smithi143 ceph-mon[36013]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:42 smithi106 ceph-mon[35889]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:42 smithi072 ceph-mon[31695]: pgmap v109: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:44 smithi106 ceph-mon[35889]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:44 smithi072 ceph-mon[31695]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:44 smithi143 ceph-mon[36013]: pgmap v110: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:46.741 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:46 smithi143 ceph-mon[36013]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:46 smithi106 ceph-mon[35889]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:46 smithi072 ceph-mon[31695]: pgmap v111: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:48 smithi106 ceph-mon[35889]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:26:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:48 smithi072 ceph-mon[31695]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:26:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:48 smithi143 ceph-mon[36013]: pgmap v112: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:26:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:50 smithi106 ceph-mon[35889]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:50 smithi072 ceph-mon[31695]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:50 smithi143 ceph-mon[36013]: pgmap v113: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:52 smithi106 ceph-mon[35889]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:52 smithi072 ceph-mon[31695]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:52.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:52 smithi143 ceph-mon[36013]: pgmap v114: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:26:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:54 smithi106 ceph-mon[35889]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:54 smithi072 ceph-mon[31695]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:54.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:54 smithi143 ceph-mon[36013]: pgmap v115: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:26:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:56 smithi106 ceph-mon[35889]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:56 smithi072 ceph-mon[31695]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:56.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:56 smithi143 ceph-mon[36013]: pgmap v116: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:26:58.733 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:26:58 smithi072 ceph-mon[31695]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:26:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:26:58 smithi106 ceph-mon[35889]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:26:58.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:26:58 smithi143 ceph-mon[36013]: pgmap v117: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:27:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:00 smithi106 ceph-mon[35889]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:00 smithi072 ceph-mon[31695]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:00.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:00 smithi143 ceph-mon[36013]: pgmap v118: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:02 smithi106 ceph-mon[35889]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:02 smithi072 ceph-mon[31695]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:02.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:02 smithi143 ceph-mon[36013]: pgmap v119: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:04 smithi106 ceph-mon[35889]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:04 smithi072 ceph-mon[31695]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:04 smithi143 ceph-mon[36013]: pgmap v120: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:06 smithi106 ceph-mon[35889]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-11T19:27:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:06 smithi072 ceph-mon[31695]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-11T19:27:06.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:06 smithi143 ceph-mon[36013]: pgmap v121: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-11T19:27:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:08 smithi106 ceph-mon[35889]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:08 smithi072 ceph-mon[31695]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:08 smithi143 ceph-mon[36013]: pgmap v122: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:10 smithi106 ceph-mon[35889]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:10 smithi072 ceph-mon[31695]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:10 smithi143 ceph-mon[36013]: pgmap v123: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:12 smithi106 ceph-mon[35889]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:12 smithi072 ceph-mon[31695]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:12 smithi143 ceph-mon[36013]: pgmap v124: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:14 smithi106 ceph-mon[35889]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:14 smithi072 ceph-mon[31695]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:14 smithi143 ceph-mon[36013]: pgmap v125: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:27:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:16 smithi106 ceph-mon[35889]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:16 smithi072 ceph-mon[31695]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:16 smithi143 ceph-mon[36013]: pgmap v126: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:27:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:18 smithi106 ceph-mon[35889]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-11T19:27:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:18 smithi072 ceph-mon[31695]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-11T19:27:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:18 smithi143 ceph-mon[36013]: pgmap v127: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-11T19:27:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:20 smithi106 ceph-mon[35889]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:27:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:20 smithi072 ceph-mon[31695]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:27:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:20 smithi143 ceph-mon[36013]: pgmap v128: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:27:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:22 smithi106 ceph-mon[35889]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:27:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:22 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:27:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:22 smithi072 ceph-mon[31695]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:27:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:22 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:27:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:22 smithi143 ceph-mon[36013]: pgmap v129: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:27:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:22 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:27:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:24 smithi106 ceph-mon[35889]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:24 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:27:24.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:24 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:27:24.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:24 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:27:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:24 smithi072 ceph-mon[31695]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:24 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:27:24.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:24 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:27:24.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:24 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:27:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:24 smithi143 ceph-mon[36013]: pgmap v130: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:24 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:27:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:24 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:27:24.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:24 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:27:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:26 smithi106 ceph-mon[35889]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:26 smithi072 ceph-mon[31695]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:26 smithi143 ceph-mon[36013]: pgmap v131: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:28 smithi106 ceph-mon[35889]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:28 smithi072 ceph-mon[31695]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:28 smithi143 ceph-mon[36013]: pgmap v132: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:30 smithi106 ceph-mon[35889]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:30 smithi072 ceph-mon[31695]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:30 smithi143 ceph-mon[36013]: pgmap v133: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:32 smithi106 ceph-mon[35889]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:32 smithi072 ceph-mon[31695]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:32 smithi143 ceph-mon[36013]: pgmap v134: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:34 smithi106 ceph-mon[35889]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:34 smithi072 ceph-mon[31695]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:34 smithi143 ceph-mon[36013]: pgmap v135: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:35 smithi106 ceph-mon[35889]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:35.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:35 smithi072 ceph-mon[31695]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:35.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:35 smithi143 ceph-mon[36013]: pgmap v136: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:38 smithi106 ceph-mon[35889]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:38 smithi072 ceph-mon[31695]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:38.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:38 smithi143 ceph-mon[36013]: pgmap v137: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:40 smithi106 ceph-mon[35889]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:40 smithi072 ceph-mon[31695]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:40 smithi143 ceph-mon[36013]: pgmap v138: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:42 smithi106 ceph-mon[35889]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:42 smithi072 ceph-mon[31695]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:42 smithi143 ceph-mon[36013]: pgmap v139: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:44.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:44 smithi106 ceph-mon[35889]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:44 smithi072 ceph-mon[31695]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:44 smithi143 ceph-mon[36013]: pgmap v140: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:46 smithi106 ceph-mon[35889]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:46.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:46 smithi106 ceph-mon[35889]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:27:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:46 smithi072 ceph-mon[31695]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:46 smithi072 ceph-mon[31695]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:27:46.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:46 smithi143 ceph-mon[36013]: pgmap v141: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:46.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:46 smithi143 ceph-mon[36013]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:27:48.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:48 smithi106 ceph-mon[35889]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:48 smithi072 ceph-mon[31695]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:48 smithi143 ceph-mon[36013]: pgmap v142: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:50 smithi106 ceph-mon[35889]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:50 smithi072 ceph-mon[31695]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:50 smithi143 ceph-mon[36013]: pgmap v143: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:52 smithi106 ceph-mon[35889]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:52 smithi072 ceph-mon[31695]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:52.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:52 smithi143 ceph-mon[36013]: pgmap v144: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:54.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:54 smithi106 ceph-mon[35889]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:54 smithi072 ceph-mon[31695]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:54.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:54 smithi143 ceph-mon[36013]: pgmap v145: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:56 smithi106 ceph-mon[35889]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:56 smithi072 ceph-mon[31695]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:56.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:56 smithi143 ceph-mon[36013]: pgmap v146: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:27:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:27:58 smithi106 ceph-mon[35889]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:58.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:27:58 smithi072 ceph-mon[31695]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:27:58.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:27:58 smithi143 ceph-mon[36013]: pgmap v147: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:00 smithi106 ceph-mon[35889]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:00 smithi072 ceph-mon[31695]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:00.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:00 smithi143 ceph-mon[36013]: pgmap v148: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:02 smithi106 ceph-mon[35889]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:02 smithi072 ceph-mon[31695]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:02.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:02 smithi143 ceph-mon[36013]: pgmap v149: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:04 smithi106 ceph-mon[35889]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:04 smithi072 ceph-mon[31695]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:04 smithi143 ceph-mon[36013]: pgmap v150: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:06 smithi106 ceph-mon[35889]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:06 smithi072 ceph-mon[31695]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:06.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:06 smithi143 ceph-mon[36013]: pgmap v151: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:08.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:08 smithi106 ceph-mon[35889]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:08 smithi072 ceph-mon[31695]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:08 smithi143 ceph-mon[36013]: pgmap v152: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:10 smithi106 ceph-mon[35889]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:10 smithi072 ceph-mon[31695]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:10 smithi143 ceph-mon[36013]: pgmap v153: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:12 smithi106 ceph-mon[35889]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:12 smithi072 ceph-mon[31695]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:12 smithi143 ceph-mon[36013]: pgmap v154: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:14 smithi106 ceph-mon[35889]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:14 smithi072 ceph-mon[31695]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:14 smithi143 ceph-mon[36013]: pgmap v155: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:28:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:16 smithi106 ceph-mon[35889]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:16 smithi072 ceph-mon[31695]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:16 smithi143 ceph-mon[36013]: pgmap v156: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:28:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:18 smithi106 ceph-mon[35889]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:18 smithi072 ceph-mon[31695]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:18 smithi143 ceph-mon[36013]: pgmap v157: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:19.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:19 smithi106 ceph-mon[35889]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:19.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:19 smithi072 ceph-mon[31695]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:19.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:19 smithi143 ceph-mon[36013]: pgmap v158: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:28:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:22 smithi106 ceph-mon[35889]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:28:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:22 smithi072 ceph-mon[31695]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:28:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:22 smithi143 ceph-mon[36013]: pgmap v159: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:28:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:24 smithi106 ceph-mon[35889]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:24 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:28:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:24 smithi072 ceph-mon[31695]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:24 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:28:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:24 smithi143 ceph-mon[36013]: pgmap v160: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:24 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:28:25.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:25 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:28:25.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:25 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:28:25.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:25 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:28:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:25 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:28:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:25 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:28:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:25 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:28:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:25 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:28:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:25 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:28:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:25 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:28:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:26 smithi106 ceph-mon[35889]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:26 smithi072 ceph-mon[31695]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:26 smithi143 ceph-mon[36013]: pgmap v161: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:28 smithi106 ceph-mon[35889]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:28 smithi072 ceph-mon[31695]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:28 smithi143 ceph-mon[36013]: pgmap v162: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:30 smithi106 ceph-mon[35889]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:30 smithi072 ceph-mon[31695]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:30 smithi143 ceph-mon[36013]: pgmap v163: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:32 smithi106 ceph-mon[35889]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:32 smithi072 ceph-mon[31695]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:32 smithi143 ceph-mon[36013]: pgmap v164: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:34 smithi106 ceph-mon[35889]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:34 smithi072 ceph-mon[31695]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:34 smithi143 ceph-mon[36013]: pgmap v165: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:36 smithi106 ceph-mon[35889]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:36 smithi072 ceph-mon[31695]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:36 smithi143 ceph-mon[36013]: pgmap v166: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:38 smithi106 ceph-mon[35889]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:38 smithi072 ceph-mon[31695]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:38 smithi143 ceph-mon[36013]: pgmap v167: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:40 smithi106 ceph-mon[35889]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:40 smithi072 ceph-mon[31695]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:40 smithi143 ceph-mon[36013]: pgmap v168: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:42 smithi106 ceph-mon[35889]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:42 smithi072 ceph-mon[31695]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:42 smithi143 ceph-mon[36013]: pgmap v169: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:44 smithi106 ceph-mon[35889]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:44 smithi072 ceph-mon[31695]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:44 smithi143 ceph-mon[36013]: pgmap v170: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:46 smithi106 ceph-mon[35889]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:46 smithi072 ceph-mon[31695]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:46.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:46 smithi143 ceph-mon[36013]: pgmap v171: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:48 smithi106 ceph-mon[35889]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:48 smithi072 ceph-mon[31695]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:48.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:48 smithi143 ceph-mon[36013]: pgmap v172: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:50 smithi106 ceph-mon[35889]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:50 smithi072 ceph-mon[31695]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:50 smithi143 ceph-mon[36013]: pgmap v173: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:52 smithi106 ceph-mon[35889]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:52 smithi072 ceph-mon[31695]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:52.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:52 smithi143 ceph-mon[36013]: pgmap v174: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:54 smithi106 ceph-mon[35889]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:54 smithi072 ceph-mon[31695]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:54.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:54 smithi143 ceph-mon[36013]: pgmap v175: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:56 smithi106 ceph-mon[35889]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:56 smithi072 ceph-mon[31695]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:56.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:56 smithi143 ceph-mon[36013]: pgmap v176: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:28:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:58 smithi106 ceph-mon[35889]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:58.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:58 smithi072 ceph-mon[31695]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:58.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:58 smithi143 ceph-mon[36013]: pgmap v177: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:59.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:28:59 smithi106 ceph-mon[35889]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:59.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:28:59 smithi072 ceph-mon[31695]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:28:59.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:28:59 smithi143 ceph-mon[36013]: pgmap v178: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:02 smithi106 ceph-mon[35889]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:02 smithi072 ceph-mon[31695]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:02.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:02 smithi143 ceph-mon[36013]: pgmap v179: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:04 smithi106 ceph-mon[35889]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:04 smithi072 ceph-mon[31695]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:04 smithi143 ceph-mon[36013]: pgmap v180: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:06 smithi106 ceph-mon[35889]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:06 smithi072 ceph-mon[31695]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:06.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:06 smithi143 ceph-mon[36013]: pgmap v181: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:08 smithi106 ceph-mon[35889]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:08 smithi072 ceph-mon[31695]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:08 smithi143 ceph-mon[36013]: pgmap v182: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:10 smithi106 ceph-mon[35889]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:10 smithi072 ceph-mon[31695]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:10 smithi143 ceph-mon[36013]: pgmap v183: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:12 smithi106 ceph-mon[35889]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:12 smithi072 ceph-mon[31695]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:12 smithi143 ceph-mon[36013]: pgmap v184: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:14 smithi106 ceph-mon[35889]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:14 smithi072 ceph-mon[31695]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:14 smithi143 ceph-mon[36013]: pgmap v185: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-11T19:29:16.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:16 smithi106 ceph-mon[35889]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:16 smithi072 ceph-mon[31695]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:16 smithi143 ceph-mon[36013]: pgmap v186: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-11T19:29:18.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:18 smithi106 ceph-mon[35889]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:18 smithi072 ceph-mon[31695]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:18 smithi143 ceph-mon[36013]: pgmap v187: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:20 smithi106 ceph-mon[35889]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:20 smithi072 ceph-mon[31695]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:20 smithi143 ceph-mon[36013]: pgmap v188: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-11T19:29:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:22 smithi106 ceph-mon[35889]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:29:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:22 smithi072 ceph-mon[31695]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:29:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:22 smithi143 ceph-mon[36013]: pgmap v189: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:29:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:24 smithi106 ceph-mon[35889]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:24 smithi072 ceph-mon[31695]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:24 smithi143 ceph-mon[36013]: pgmap v190: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:25.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:25 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:25 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:25.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:25 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:26.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:26 smithi106 ceph-mon[35889]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:26 smithi072 ceph-mon[31695]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:26 smithi143 ceph-mon[36013]: pgmap v191: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:27 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:29:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:27 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:29:27.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:27 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:29:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:27 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:29:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:27 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:29:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:27 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:29:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:27 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:29:27.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:27 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:29:27.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:27 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:29:28.832 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:28 smithi072 ceph-mon[31695]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:28 smithi106 ceph-mon[35889]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:28 smithi143 ceph-mon[36013]: pgmap v192: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:30 smithi106 ceph-mon[35889]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:30 smithi072 ceph-mon[31695]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:30.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:30 smithi143 ceph-mon[36013]: pgmap v193: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:32 smithi106 ceph-mon[35889]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:32 smithi072 ceph-mon[31695]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:32 smithi143 ceph-mon[36013]: pgmap v194: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:34.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:34 smithi106 ceph-mon[35889]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:34 smithi072 ceph-mon[31695]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:34 smithi143 ceph-mon[36013]: pgmap v195: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:36 smithi106 ceph-mon[35889]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:36 smithi072 ceph-mon[31695]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:36 smithi143 ceph-mon[36013]: pgmap v196: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:37.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:37 smithi106 ceph-mon[35889]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:37 smithi072 ceph-mon[31695]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:37.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:37 smithi143 ceph-mon[36013]: pgmap v197: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:40.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:40 smithi106 ceph-mon[35889]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:40 smithi072 ceph-mon[31695]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:40 smithi143 ceph-mon[36013]: pgmap v198: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:42 smithi106 ceph-mon[35889]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:42 smithi072 ceph-mon[31695]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:42 smithi143 ceph-mon[36013]: pgmap v199: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:44 smithi106 ceph-mon[35889]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:44 smithi072 ceph-mon[31695]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:44 smithi143 ceph-mon[36013]: pgmap v200: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:46.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:46 smithi106 ceph-mon[35889]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:46 smithi072 ceph-mon[31695]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:46.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:46 smithi143 ceph-mon[36013]: pgmap v201: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:48 smithi106 ceph-mon[35889]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:48 smithi072 ceph-mon[31695]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:48 smithi143 ceph-mon[36013]: pgmap v202: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:50 smithi106 ceph-mon[35889]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:50 smithi072 ceph-mon[31695]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:50 smithi143 ceph-mon[36013]: pgmap v203: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:52.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:52 smithi106 ceph-mon[35889]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:52 smithi072 ceph-mon[31695]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:52.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:52 smithi143 ceph-mon[36013]: pgmap v204: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:54 smithi106 ceph-mon[35889]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:54 smithi072 ceph-mon[31695]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:54.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:54 smithi143 ceph-mon[36013]: pgmap v205: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:56 smithi106 ceph-mon[35889]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:56 smithi072 ceph-mon[31695]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:56.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:56 smithi143 ceph-mon[36013]: pgmap v206: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:29:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:29:58 smithi106 ceph-mon[35889]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:58.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:29:58 smithi072 ceph-mon[31695]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:29:58.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:29:58 smithi143 ceph-mon[36013]: pgmap v207: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:00 smithi106 ceph-mon[35889]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:00.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:00 smithi106 ceph-mon[35889]: overall HEALTH_OK 2024-03-11T19:30:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:00 smithi072 ceph-mon[31695]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:00 smithi072 ceph-mon[31695]: overall HEALTH_OK 2024-03-11T19:30:00.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:00 smithi143 ceph-mon[36013]: pgmap v208: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:00.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:00 smithi143 ceph-mon[36013]: overall HEALTH_OK 2024-03-11T19:30:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:02 smithi106 ceph-mon[35889]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:02 smithi072 ceph-mon[31695]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:02.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:02 smithi143 ceph-mon[36013]: pgmap v209: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:04 smithi106 ceph-mon[35889]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:04 smithi072 ceph-mon[31695]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:04 smithi143 ceph-mon[36013]: pgmap v210: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:06 smithi106 ceph-mon[35889]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:30:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:06 smithi072 ceph-mon[31695]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:30:06.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:06 smithi143 ceph-mon[36013]: pgmap v211: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:30:07.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:07 smithi106 ceph-mon[35889]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:30:07.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:07 smithi072 ceph-mon[31695]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:30:07.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:07 smithi143 ceph-mon[36013]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:30:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:08 smithi106 ceph-mon[35889]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:08 smithi072 ceph-mon[31695]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:08.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:08 smithi143 ceph-mon[36013]: pgmap v213: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:09.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:09 smithi106 ceph-mon[35889]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:09.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:09 smithi072 ceph-mon[31695]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:09.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:09 smithi143 ceph-mon[36013]: pgmap v214: 129 pgs: 129 active+clean; 20 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:12 smithi106 ceph-mon[35889]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:12 smithi106 ceph-mon[35889]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:30:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:12 smithi072 ceph-mon[31695]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:12 smithi072 ceph-mon[31695]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:30:12.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:12 smithi143 ceph-mon[36013]: pgmap v215: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:30:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:12 smithi143 ceph-mon[36013]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:30:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:14 smithi106 ceph-mon[35889]: pgmap v217: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:30:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:14 smithi072 ceph-mon[31695]: pgmap v217: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:30:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:14 smithi143 ceph-mon[36013]: pgmap v217: 129 pgs: 129 active+clean; 20 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:30:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:16 smithi106 ceph-mon[35889]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-03-11T19:30:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:16 smithi072 ceph-mon[31695]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-03-11T19:30:16.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:16 smithi143 ceph-mon[36013]: pgmap v218: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-03-11T19:30:18.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:18 smithi106 ceph-mon[35889]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:18 smithi072 ceph-mon[31695]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:18.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:18 smithi143 ceph-mon[36013]: pgmap v219: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:20 smithi106 ceph-mon[35889]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:20 smithi072 ceph-mon[31695]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:20 smithi143 ceph-mon[36013]: pgmap v220: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:30:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:22 smithi106 ceph-mon[35889]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:30:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:22 smithi072 ceph-mon[31695]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:30:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:22 smithi143 ceph-mon[36013]: pgmap v221: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:30:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:24 smithi106 ceph-mon[35889]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:30:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:24 smithi072 ceph-mon[31695]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:30:24.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:24 smithi143 ceph-mon[36013]: pgmap v222: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:30:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:26 smithi106 ceph-mon[35889]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:30:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:26 smithi072 ceph-mon[31695]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:30:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:26 smithi143 ceph-mon[36013]: pgmap v223: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:30:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:27 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:27.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:27 smithi106 ceph-mon[35889]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:27 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:27 smithi072 ceph-mon[31695]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:27.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:27 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:27.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:27 smithi143 ceph-mon[36013]: pgmap v224: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:28 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:28.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:28 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:28.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:28 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:30:28.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:28 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:28.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:28 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:28.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:28 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:30:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:28 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:28 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:28.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:28 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:30:29.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:29 smithi072 ceph-mon[31695]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:29.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:29 smithi143 ceph-mon[36013]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:30.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:29 smithi106 ceph-mon[35889]: pgmap v225: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:32 smithi106 ceph-mon[35889]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:32 smithi072 ceph-mon[31695]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:32 smithi143 ceph-mon[36013]: pgmap v226: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:34 smithi106 ceph-mon[35889]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:34 smithi072 ceph-mon[31695]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:34 smithi143 ceph-mon[36013]: pgmap v227: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T19:30:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:36 smithi106 ceph-mon[35889]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:36.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:36 smithi072 ceph-mon[31695]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:36.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:36 smithi143 ceph-mon[36013]: pgmap v228: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:38 smithi106 ceph-mon[35889]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:38 smithi072 ceph-mon[31695]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:38 smithi143 ceph-mon[36013]: pgmap v229: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:40 smithi106 ceph-mon[35889]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:40 smithi072 ceph-mon[31695]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:40.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:40 smithi143 ceph-mon[36013]: pgmap v230: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:42 smithi106 ceph-mon[35889]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:42 smithi072 ceph-mon[31695]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:42 smithi143 ceph-mon[36013]: pgmap v231: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:44 smithi106 ceph-mon[35889]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:44 smithi072 ceph-mon[31695]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:44 smithi143 ceph-mon[36013]: pgmap v232: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:46 smithi106 ceph-mon[35889]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:46 smithi072 ceph-mon[31695]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:46.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:46 smithi143 ceph-mon[36013]: pgmap v233: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:48 smithi106 ceph-mon[35889]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:48 smithi072 ceph-mon[31695]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:48.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:48 smithi143 ceph-mon[36013]: pgmap v234: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:50.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:50 smithi106 ceph-mon[35889]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:50 smithi072 ceph-mon[31695]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:50 smithi143 ceph-mon[36013]: pgmap v235: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:52 smithi106 ceph-mon[35889]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:52 smithi072 ceph-mon[31695]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:52.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:52 smithi143 ceph-mon[36013]: pgmap v236: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:54 smithi106 ceph-mon[35889]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:54 smithi072 ceph-mon[31695]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:54.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:54 smithi143 ceph-mon[36013]: pgmap v237: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:56.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:56 smithi106 ceph-mon[35889]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:56 smithi072 ceph-mon[31695]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:56.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:56 smithi143 ceph-mon[36013]: pgmap v238: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:30:57.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:30:57 smithi072 ceph-mon[31695]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:57.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:30:57 smithi143 ceph-mon[36013]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:30:58.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:30:57 smithi106 ceph-mon[35889]: pgmap v239: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:00.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:00 smithi106 ceph-mon[35889]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:00 smithi072 ceph-mon[31695]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:00.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:00 smithi143 ceph-mon[36013]: pgmap v240: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:02 smithi106 ceph-mon[35889]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:02.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:02 smithi072 ceph-mon[31695]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:02.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:02 smithi143 ceph-mon[36013]: pgmap v241: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:04.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:04 smithi106 ceph-mon[35889]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T19:31:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:04 smithi072 ceph-mon[31695]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T19:31:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:04 smithi143 ceph-mon[36013]: pgmap v242: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T19:31:06.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:06 smithi106 ceph-mon[35889]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:06 smithi072 ceph-mon[31695]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:06.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:06 smithi143 ceph-mon[36013]: pgmap v243: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:31:07.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:07 smithi106 ceph-mon[35889]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T19:31:07.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:07 smithi072 ceph-mon[31695]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T19:31:07.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:07 smithi143 ceph-mon[36013]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T19:31:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:08 smithi106 ceph-mon[35889]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:08 smithi072 ceph-mon[31695]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:08 smithi143 ceph-mon[36013]: pgmap v245: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:10 smithi106 ceph-mon[35889]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:10 smithi072 ceph-mon[31695]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:10.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:10 smithi143 ceph-mon[36013]: pgmap v246: 129 pgs: 129 active+clean; 20 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:31:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:12 smithi106 ceph-mon[35889]: pgmap v247: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:12 smithi106 ceph-mon[35889]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T19:31:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:12 smithi072 ceph-mon[31695]: pgmap v247: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:12 smithi072 ceph-mon[31695]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T19:31:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:12 smithi143 ceph-mon[36013]: pgmap v247: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:12 smithi143 ceph-mon[36013]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T19:31:13.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:13 smithi072 ceph-mon[31695]: pgmap v249: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:31:13.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:13 smithi143 ceph-mon[36013]: pgmap v249: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:31:14.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:13 smithi106 ceph-mon[35889]: pgmap v249: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:31:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:16 smithi106 ceph-mon[35889]: pgmap v250: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:31:16.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:16 smithi072 ceph-mon[31695]: pgmap v250: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:31:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:16 smithi143 ceph-mon[36013]: pgmap v250: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:31:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:18 smithi106 ceph-mon[35889]: pgmap v251: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:18.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:18 smithi072 ceph-mon[31695]: pgmap v251: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:18.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:18 smithi143 ceph-mon[36013]: pgmap v251: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:20.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:20 smithi106 ceph-mon[35889]: pgmap v252: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:20 smithi072 ceph-mon[31695]: pgmap v252: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:20.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:20 smithi143 ceph-mon[36013]: pgmap v252: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:31:22.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:22 smithi106 ceph-mon[35889]: pgmap v253: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:31:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:22 smithi072 ceph-mon[31695]: pgmap v253: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:31:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:22 smithi143 ceph-mon[36013]: pgmap v253: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:31:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:24 smithi106 ceph-mon[35889]: pgmap v254: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:31:24.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:24 smithi072 ceph-mon[31695]: pgmap v254: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:31:24.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:24 smithi143 ceph-mon[36013]: pgmap v254: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:31:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:26 smithi106 ceph-mon[35889]: pgmap v255: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:31:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:26 smithi072 ceph-mon[31695]: pgmap v255: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:31:26.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:26 smithi143 ceph-mon[36013]: pgmap v255: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:31:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:28 smithi106 ceph-mon[35889]: pgmap v256: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:28 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:28 smithi072 ceph-mon[31695]: pgmap v256: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:28.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:28 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:28 smithi143 ceph-mon[36013]: pgmap v256: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:28.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:28 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:30 smithi106 ceph-mon[35889]: pgmap v257: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:30.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:30 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:30.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:30 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:30.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:30 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:31:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:30 smithi072 ceph-mon[31695]: pgmap v257: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:30.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:30 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:30.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:30 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:30.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:30 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:31:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:30 smithi143 ceph-mon[36013]: pgmap v257: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:30 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:30 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:30 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:31:32.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:32 smithi106 ceph-mon[35889]: pgmap v258: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:32 smithi072 ceph-mon[31695]: pgmap v258: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:32 smithi143 ceph-mon[36013]: pgmap v258: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:34 smithi106 ceph-mon[35889]: pgmap v259: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:34 smithi072 ceph-mon[31695]: pgmap v259: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:34 smithi143 ceph-mon[36013]: pgmap v259: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:35.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:35 smithi072 ceph-mon[31695]: pgmap v260: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:35.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:35 smithi143 ceph-mon[36013]: pgmap v260: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:36.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:35 smithi106 ceph-mon[35889]: pgmap v260: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:38 smithi106 ceph-mon[35889]: pgmap v261: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:38 smithi072 ceph-mon[31695]: pgmap v261: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:38 smithi143 ceph-mon[36013]: pgmap v261: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:40.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:40 smithi106 ceph-mon[35889]: pgmap v262: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:40 smithi072 ceph-mon[31695]: pgmap v262: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:40.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:40 smithi143 ceph-mon[36013]: pgmap v262: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:42.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:42 smithi106 ceph-mon[35889]: pgmap v263: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:42.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:42 smithi072 ceph-mon[31695]: pgmap v263: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:42.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:42 smithi143 ceph-mon[36013]: pgmap v263: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:44.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:44 smithi106 ceph-mon[35889]: pgmap v264: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:44 smithi072 ceph-mon[31695]: pgmap v264: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:44.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:44 smithi143 ceph-mon[36013]: pgmap v264: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:46 smithi106 ceph-mon[35889]: pgmap v265: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:46 smithi072 ceph-mon[31695]: pgmap v265: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:46.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:46 smithi143 ceph-mon[36013]: pgmap v265: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:48.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:48 smithi106 ceph-mon[35889]: pgmap v266: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:48 smithi072 ceph-mon[31695]: pgmap v266: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:48.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:48 smithi143 ceph-mon[36013]: pgmap v266: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:50.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:50 smithi106 ceph-mon[35889]: pgmap v267: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:50 smithi072 ceph-mon[31695]: pgmap v267: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:50 smithi143 ceph-mon[36013]: pgmap v267: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:52.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:52 smithi106 ceph-mon[35889]: pgmap v268: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:52 smithi072 ceph-mon[31695]: pgmap v268: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:52.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:52 smithi143 ceph-mon[36013]: pgmap v268: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:54.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:54 smithi106 ceph-mon[35889]: pgmap v269: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:54.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:54 smithi072 ceph-mon[31695]: pgmap v269: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:54.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:54 smithi143 ceph-mon[36013]: pgmap v269: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:55.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:55 smithi106 ceph-mon[35889]: pgmap v270: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:55.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:55 smithi072 ceph-mon[31695]: pgmap v270: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:55.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:55 smithi143 ceph-mon[36013]: pgmap v270: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:31:58.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:31:58 smithi106 ceph-mon[35889]: pgmap v271: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:58.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:31:58 smithi072 ceph-mon[31695]: pgmap v271: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:31:58.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:31:58 smithi143 ceph-mon[36013]: pgmap v271: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:00 smithi106 ceph-mon[35889]: pgmap v272: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:00 smithi072 ceph-mon[31695]: pgmap v272: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:00.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:00 smithi143 ceph-mon[36013]: pgmap v272: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:02 smithi106 ceph-mon[35889]: pgmap v273: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-03-11T19:32:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:02 smithi072 ceph-mon[31695]: pgmap v273: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-03-11T19:32:02.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:02 smithi143 ceph-mon[36013]: pgmap v273: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-03-11T19:32:04.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:04 smithi106 ceph-mon[35889]: pgmap v274: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:32:04.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:04 smithi072 ceph-mon[31695]: pgmap v274: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:32:04.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:04 smithi143 ceph-mon[36013]: pgmap v274: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-03-11T19:32:06.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:06 smithi106 ceph-mon[35889]: pgmap v275: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:32:06.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:06 smithi072 ceph-mon[31695]: pgmap v275: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:32:06.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:06 smithi143 ceph-mon[36013]: pgmap v275: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:32:07.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:07 smithi106 ceph-mon[35889]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T19:32:07.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:07 smithi106 ceph-mon[35889]: pgmap v277: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:07.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:07 smithi072 ceph-mon[31695]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T19:32:07.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:07 smithi072 ceph-mon[31695]: pgmap v277: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:07.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:07 smithi143 ceph-mon[36013]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T19:32:07.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:07 smithi143 ceph-mon[36013]: pgmap v277: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:10.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:10 smithi106 ceph-mon[35889]: pgmap v278: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:10.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:10 smithi072 ceph-mon[31695]: pgmap v278: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:10.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:10 smithi143 ceph-mon[36013]: pgmap v278: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:32:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:12 smithi106 ceph-mon[35889]: pgmap v279: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-03-11T19:32:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:12 smithi106 ceph-mon[35889]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T19:32:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:12 smithi072 ceph-mon[31695]: pgmap v279: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-03-11T19:32:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:12 smithi072 ceph-mon[31695]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T19:32:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:12 smithi143 ceph-mon[36013]: pgmap v279: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-03-11T19:32:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:12 smithi143 ceph-mon[36013]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T19:32:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:14 smithi106 ceph-mon[35889]: pgmap v281: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:32:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:14 smithi072 ceph-mon[31695]: pgmap v281: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:32:14.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:14 smithi143 ceph-mon[36013]: pgmap v281: 129 pgs: 129 active+clean; 16 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:32:16.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:16 smithi106 ceph-mon[35889]: pgmap v282: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:32:16.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:16 smithi072 ceph-mon[31695]: pgmap v282: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:32:16.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:16 smithi143 ceph-mon[36013]: pgmap v282: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2024-03-11T19:32:18.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:18 smithi106 ceph-mon[35889]: pgmap v283: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:18.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:18 smithi072 ceph-mon[31695]: pgmap v283: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:18.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:18 smithi143 ceph-mon[36013]: pgmap v283: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:20 smithi106 ceph-mon[35889]: pgmap v284: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:20 smithi072 ceph-mon[31695]: pgmap v284: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:20.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:20 smithi143 ceph-mon[36013]: pgmap v284: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T19:32:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:22 smithi106 ceph-mon[35889]: pgmap v285: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:32:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:22 smithi072 ceph-mon[31695]: pgmap v285: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:32:22.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:22 smithi143 ceph-mon[36013]: pgmap v285: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T19:32:23.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:23 smithi106 ceph-mon[35889]: pgmap v286: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:32:23.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:23 smithi072 ceph-mon[31695]: pgmap v286: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:32:23.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:23 smithi143 ceph-mon[36013]: pgmap v286: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 86 B/s wr, 5 op/s 2024-03-11T19:32:26.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:26 smithi106 ceph-mon[35889]: pgmap v287: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:32:26.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:26 smithi072 ceph-mon[31695]: pgmap v287: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:32:26.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:26 smithi143 ceph-mon[36013]: pgmap v287: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-11T19:32:27.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:27 smithi072 ceph-mon[31695]: pgmap v288: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:27.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:27 smithi143 ceph-mon[36013]: pgmap v288: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:28.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:27 smithi106 ceph-mon[35889]: pgmap v288: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:30.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:30 smithi106 ceph-mon[35889]: pgmap v289: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:30.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:30 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:30.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:30 smithi072 ceph-mon[31695]: pgmap v289: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:30.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:30 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:30 smithi143 ceph-mon[36013]: pgmap v289: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:30.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:30 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:31.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:31 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:31.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:31 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:31.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:31 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:32:31.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:31 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:31.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:31 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:31.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:31 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:32:31.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:31 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:31.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:31 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:31.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:31 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:32:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:32 smithi106 ceph-mon[35889]: pgmap v290: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:32.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:32 smithi072 ceph-mon[31695]: pgmap v290: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:32 smithi143 ceph-mon[36013]: pgmap v290: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:34.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:34 smithi106 ceph-mon[35889]: pgmap v291: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:34.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:34 smithi072 ceph-mon[31695]: pgmap v291: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:34.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:34 smithi143 ceph-mon[36013]: pgmap v291: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:36 smithi106 ceph-mon[35889]: pgmap v292: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:36 smithi072 ceph-mon[31695]: pgmap v292: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:36 smithi143 ceph-mon[36013]: pgmap v292: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:38.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:38 smithi106 ceph-mon[35889]: pgmap v293: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:38.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:38 smithi072 ceph-mon[31695]: pgmap v293: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:38.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:38 smithi143 ceph-mon[36013]: pgmap v293: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:40.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:40 smithi106 ceph-mon[35889]: pgmap v294: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:40.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:40 smithi072 ceph-mon[31695]: pgmap v294: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:40.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:40 smithi143 ceph-mon[36013]: pgmap v294: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:41.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:41 smithi106 ceph-mon[35889]: pgmap v295: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:41.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:41 smithi072 ceph-mon[31695]: pgmap v295: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:41.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:41 smithi143 ceph-mon[36013]: pgmap v295: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:44.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:44 smithi106 ceph-mon[35889]: pgmap v296: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:44.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:44 smithi072 ceph-mon[31695]: pgmap v296: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:44.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:44 smithi143 ceph-mon[36013]: pgmap v296: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:46.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:46 smithi106 ceph-mon[35889]: pgmap v297: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:46 smithi072 ceph-mon[31695]: pgmap v297: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:46.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:46 smithi143 ceph-mon[36013]: pgmap v297: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:48.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:48 smithi106 ceph-mon[35889]: pgmap v298: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:48.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:48 smithi072 ceph-mon[31695]: pgmap v298: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:48.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:48 smithi143 ceph-mon[36013]: pgmap v298: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:50.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:50 smithi106 ceph-mon[35889]: pgmap v299: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:50 smithi072 ceph-mon[31695]: pgmap v299: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:50.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:50 smithi143 ceph-mon[36013]: pgmap v299: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:52.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:52 smithi106 ceph-mon[35889]: pgmap v300: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:52.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:52 smithi072 ceph-mon[31695]: pgmap v300: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:52.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:52 smithi143 ceph-mon[36013]: pgmap v300: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:53.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:53 smithi106 ceph-mon[35889]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:53.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:53 smithi072 ceph-mon[31695]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:53.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:53 smithi143 ceph-mon[36013]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:56.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:56 smithi106 ceph-mon[35889]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:56.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:56 smithi072 ceph-mon[31695]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:56.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:56 smithi143 ceph-mon[36013]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:32:58.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:32:58 smithi106 ceph-mon[35889]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:58.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:32:58 smithi072 ceph-mon[31695]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:32:58.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:32:58 smithi143 ceph-mon[36013]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:00.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:00 smithi106 ceph-mon[35889]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:00.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:00 smithi072 ceph-mon[31695]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:00.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:00 smithi143 ceph-mon[36013]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:02.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:02 smithi106 ceph-mon[35889]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:02.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:02 smithi072 ceph-mon[31695]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:02.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:02 smithi143 ceph-mon[36013]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:03.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:03 smithi072 ceph-mon[31695]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:03.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:03 smithi143 ceph-mon[36013]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:04.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:03 smithi106 ceph-mon[35889]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:06.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:06 smithi106 ceph-mon[35889]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:33:06.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:06 smithi072 ceph-mon[31695]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:33:06.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:06 smithi143 ceph-mon[36013]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:33:07.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:07 smithi072 ceph-mon[31695]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T19:33:07.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:07 smithi143 ceph-mon[36013]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T19:33:08.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:07 smithi106 ceph-mon[35889]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T19:33:08.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:08 smithi072 ceph-mon[31695]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:08 smithi143 ceph-mon[36013]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:09.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:08 smithi106 ceph-mon[35889]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:09.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:09 smithi072 ceph-mon[31695]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:09.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:09 smithi143 ceph-mon[36013]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:10.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:09 smithi106 ceph-mon[35889]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:12.309 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 -- 172.21.15.72:0/3472843465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 msgr2=0x7fca1c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 --2- 172.21.15.72:0/3472843465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c1015d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fca04009930 tx=0x7fca0402bc30 comp rx=0 tx=0).stop 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 -- 172.21.15.72:0/3472843465 shutdown_connections 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 --2- 172.21.15.72:0/3472843465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1062c0 0x7fca1c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 --2- 172.21.15.72:0/3472843465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c101ba0 0x7fca1c105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 --2- 172.21.15.72:0/3472843465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c1015d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 -- 172.21.15.72:0/3472843465 >> 172.21.15.72:0/3472843465 conn(0x7fca1c0fca90 msgr2=0x7fca1c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:12.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 -- 172.21.15.72:0/3472843465 shutdown_connections 2024-03-11T19:33:12.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.524+0000 7fca1b200640 1 -- 172.21.15.72:0/3472843465 wait complete. 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.525+0000 7fca1b200640 1 Processor -- start 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.525+0000 7fca1b200640 1 -- start start 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c1062c0 0x7fca1c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1996b0 0x7fca1c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19a180 con 0x7fca1c1011f0 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19a2f0 con 0x7fca1c1996b0 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19a4b0 con 0x7fca1c1062c0 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49358/0 (socket says 172.21.15.72:49358) 2024-03-11T19:33:12.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19e00640 1 -- 172.21.15.72:0/80368798 learned_addr learned my addr 172.21.15.72:0/80368798 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19400640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c1062c0 0x7fca1c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca1a800640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1996b0 0x7fca1c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19e00640 1 -- 172.21.15.72:0/80368798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c1062c0 msgr2=0x7fca1c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.526+0000 7fca19e00640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c1062c0 0x7fca1c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca19e00640 1 -- 172.21.15.72:0/80368798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1996b0 msgr2=0x7fca1c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca19e00640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1996b0 0x7fca1c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca19e00640 1 -- 172.21.15.72:0/80368798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009560 con 0x7fca1c1011f0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca19e00640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca04000c00 tx=0x7fca0402cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04004350 con 0x7fca1c1011f0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1a63f0 con 0x7fca1c1011f0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca040044b0 con 0x7fca1c1011f0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04003ca0 con 0x7fca1c1011f0 2024-03-11T19:33:12.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.527+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a68d0 con 0x7fca1c1011f0 2024-03-11T19:33:12.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.529+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c1026c0 con 0x7fca1c1011f0 2024-03-11T19:33:12.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.530+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fca04004df0 con 0x7fca1c1011f0 2024-03-11T19:33:12.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.532+0000 7fca12a00640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 0x7fca000a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:12.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.532+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fca040f0150 con 0x7fca1c1011f0 2024-03-11T19:33:12.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.532+0000 7fca19400640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 0x7fca000a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:12.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.533+0000 7fca19400640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 0x7fca000a4050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c19af00 tx=0x7fca0c009500 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:12.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.536+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca040b8de0 con 0x7fca1c1011f0 2024-03-11T19:33:12.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.759+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fca1c101ba0 con 0x7fca000a1b90 2024-03-11T19:33:12.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.775+0000 7fca12a00640 1 -- 172.21.15.72:0/80368798 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7fca1c101ba0 con 0x7fca000a1b90 2024-03-11T19:33:12.780 INFO:teuthology.orchestra.run.smithi072.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-11T00:00:00", "created": "2024-03-11T19:23:10", "first": "2024-03-11T19:24:00", "last": "2024-03-11T19:33:00", "last_pruned": "2024-03-11T19:33:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-11T19:33:12.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 msgr2=0x7fca000a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 0x7fca000a4050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca1c19af00 tx=0x7fca0c009500 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 msgr2=0x7fca1c19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca04000c00 tx=0x7fca0402cd80 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 shutdown_connections 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fca000a1b90 0x7fca000a4050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca1c1996b0 0x7fca1c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca1c1062c0 0x7fca1c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 --2- 172.21.15.72:0/80368798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca1c1011f0 0x7fca1c19f5f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 >> 172.21.15.72:0/80368798 conn(0x7fca1c0fca90 msgr2=0x7fca1c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 shutdown_connections 2024-03-11T19:33:12.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.778+0000 7fca1b200640 1 -- 172.21.15.72:0/80368798 wait complete. 2024-03-11T19:33:12.800 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-11T19:33:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:12 smithi106 ceph-mon[35889]: pgmap v311: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:12 smithi106 ceph-mon[35889]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T19:33:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:12 smithi072 ceph-mon[31695]: pgmap v311: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:12.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:12 smithi072 ceph-mon[31695]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T19:33:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:12 smithi143 ceph-mon[36013]: pgmap v311: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-11T19:33:12.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:12 smithi143 ceph-mon[36013]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T19:33:12.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 -- 172.21.15.72:0/2061757497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 msgr2=0x7f8b64114620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 --2- 172.21.15.72:0/2061757497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b64114620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009cd0 tx=0x7f8b5c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 -- 172.21.15.72:0/2061757497 shutdown_connections 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 --2- 172.21.15.72:0/2061757497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b64114620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 --2- 172.21.15.72:0/2061757497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b64107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 --2- 172.21.15.72:0/2061757497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6410db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 -- 172.21.15.72:0/2061757497 >> 172.21.15.72:0/2061757497 conn(0x7f8b640756f0 msgr2=0x7f8b64077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 -- 172.21.15.72:0/2061757497 shutdown_connections 2024-03-11T19:33:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.993+0000 7f8b68a00640 1 -- 172.21.15.72:0/2061757497 wait complete. 2024-03-11T19:33:13.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 Processor -- start 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 -- start start 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b641a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6419e2c0 con 0x7f8b64108110 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6419e430 con 0x7f8b6410d760 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.994+0000 7f8b68a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6419e5f0 con 0x7f8b64107750 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b641a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49370/0 (socket says 172.21.15.72:49370) 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61400640 1 -- 172.21.15.72:0/1019261078 learned_addr learned my addr 172.21.15.72:0/1019261078 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b641a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:59036/0 (socket says 172.21.15.72:59036) 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 -- 172.21.15.72:0/1019261078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 msgr2=0x7f8b641a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b641a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 -- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 msgr2=0x7f8b641a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b62800640 1 -- 172.21.15.72:0/1019261078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c009900 con 0x7f8b6410d760 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.995+0000 7f8b61400640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b62800640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6419db30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c037040 tx=0x7f8b5c034cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c032ac0 con 0x7f8b6410d760 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6419e870 con 0x7f8b6410d760 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c032eb0 con 0x7f8b6410d760 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c004880 con 0x7f8b6410d760 2024-03-11T19:33:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.996+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b641aa9f0 con 0x7f8b6410d760 2024-03-11T19:33:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.997+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6419ea50 con 0x7f8b6410d760 2024-03-11T19:33:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:12.999+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8b5c00a040 con 0x7f8b6410d760 2024-03-11T19:33:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.000+0000 7f8b5aa00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 0x7f8b480a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.000+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f8b5c0f0aa0 con 0x7f8b6410d760 2024-03-11T19:33:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.001+0000 7f8b61e00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 0x7f8b480a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.001+0000 7f8b61e00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 0x7f8b480a4100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400bb60 tx=0x7f8b5400b390 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.003+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b5c0b9640 con 0x7f8b6410d760 2024-03-11T19:33:13.230 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.225+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 --> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8b6419ea00 con 0x7f8b480a1c40 2024-03-11T19:33:13.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.239+0000 7f8b5aa00640 1 -- 172.21.15.72:0/1019261078 <== mgr.15303 v2:172.21.15.72:6800/3931879529 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f8b6419ea00 con 0x7f8b480a1c40 2024-03-11T19:33:13.243 INFO:teuthology.orchestra.run.smithi072.stdout:/ 1m 6m3h 2024-03-11T19:33:13.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.241+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 msgr2=0x7f8b480a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.241+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 0x7f8b480a4100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400bb60 tx=0x7f8b5400b390 comp rx=0 tx=0).stop 2024-03-11T19:33:13.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.241+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 msgr2=0x7f8b6419db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.241+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6419db30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c037040 tx=0x7f8b5c034cc0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.241+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 shutdown_connections 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f8b480a1c40 0x7f8b480a4100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b6410d760 0x7f8b6419db30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b64108110 0x7f8b641a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 --2- 172.21.15.72:0/1019261078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b64107750 0x7f8b641a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 >> 172.21.15.72:0/1019261078 conn(0x7f8b640756f0 msgr2=0x7f8b64075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 shutdown_connections 2024-03-11T19:33:13.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.242+0000 7f8b68a00640 1 -- 172.21.15.72:0/1019261078 wait complete. 2024-03-11T19:33:13.265 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-11T19:33:13.276 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:33:13.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 -- 172.21.15.72:0/606606353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884108490 msgr2=0x7f8884110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 --2- 172.21.15.72:0/606606353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884108490 0x7f8884110060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f887c009cd0 tx=0x7f887c02c0e0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 -- 172.21.15.72:0/606606353 shutdown_connections 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 --2- 172.21.15.72:0/606606353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884108490 0x7f8884110060 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 --2- 172.21.15.72:0/606606353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8884103d70 0x7f8884107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 --2- 172.21.15.72:0/606606353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88841033c0 0x7f88841037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 -- 172.21.15.72:0/606606353 >> 172.21.15.72:0/606606353 conn(0x7f88840fec20 msgr2=0x7f8884101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.488+0000 7f8888e00640 1 -- 172.21.15.72:0/606606353 shutdown_connections 2024-03-11T19:33:13.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.489+0000 7f8888e00640 1 -- 172.21.15.72:0/606606353 wait complete. 2024-03-11T19:33:13.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.489+0000 7f8888e00640 1 Processor -- start 2024-03-11T19:33:13.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 -- start start 2024-03-11T19:33:13.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8884108490 0x7f888419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 0x7f8884199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884199ff0 con 0x7f8884199600 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419a160 con 0x7f88841033c0 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8888e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888419a320 con 0x7f8884108490 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8883e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 0x7f8884199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8883400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f887be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8884108490 0x7f888419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8883400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:47390/0 (socket says 172.21.15.72:47390) 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.490+0000 7f8883400640 1 -- 172.21.15.72:0/2776499692 learned_addr learned my addr 172.21.15.72:0/2776499692 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 -- 172.21.15.72:0/2776499692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8884108490 msgr2=0x7f888419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8884108490 0x7f888419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 -- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 msgr2=0x7f8884199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 0x7f8884199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 -- 172.21.15.72:0/2776499692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c009900 con 0x7f88841033c0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 0x7f8884199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8883400640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f887400e9f0 tx=0x7f887400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874009700 con 0x7f88841033c0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88841a6310 con 0x7f88841033c0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f887400cd80 con 0x7f88841033c0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874018920 con 0x7f88841033c0 2024-03-11T19:33:13.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.491+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88841a6820 con 0x7f88841033c0 2024-03-11T19:33:13.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.492+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88840682e0 con 0x7f88841033c0 2024-03-11T19:33:13.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.494+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f8874018a80 con 0x7f88841033c0 2024-03-11T19:33:13.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.496+0000 7f8880c00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 0x7f88680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.496+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f8874014070 con 0x7f88841033c0 2024-03-11T19:33:13.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.496+0000 7f887be00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 0x7f88680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.497+0000 7f887be00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 0x7f88680a4060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8884109a60 tx=0x7f886c005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.498+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8874096fa0 con 0x7f88841033c0 2024-03-11T19:33:13.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.696+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f888419ab30 con 0x7f88841033c0 2024-03-11T19:33:13.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.699+0000 7f8880c00640 1 -- 172.21.15.72:0/2776499692 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25336 (secure 0 0 0) 0x7f887409a660 con 0x7f88841033c0 2024-03-11T19:33:13.704 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:33:13.704 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":103,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:33:11.664414+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/537254334":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1379647566":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2158720129":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/4059778386":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/828846469":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3915050930":"2024-03-12T19:23:09.241200+0000","172.21.15.72:6800/537254334":"2024-03-12T19:23:09.241200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 msgr2=0x7f88680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 0x7f88680a4060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8884109a60 tx=0x7f886c005f90 comp rx=0 tx=0).stop 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 msgr2=0x7f888419f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f887400e9f0 tx=0x7f887400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 shutdown_connections 2024-03-11T19:33:13.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f88680a1ba0 0x7f88680a4060 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8884199600 0x7f8884199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8884108490 0x7f888419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 --2- 172.21.15.72:0/2776499692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88841033c0 0x7f888419f450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.701+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 >> 172.21.15.72:0/2776499692 conn(0x7f88840fec20 msgr2=0x7f8884100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.702+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 shutdown_connections 2024-03-11T19:33:13.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.702+0000 7f8888e00640 1 -- 172.21.15.72:0/2776499692 wait complete. 2024-03-11T19:33:13.727 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:33:13.728 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 -- 172.21.15.72:0/3053870175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101d30 msgr2=0x7f1b9c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/3053870175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101d30 0x7f1b9c105d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009930 tx=0x7f1b8402bc30 comp rx=0 tx=0).stop 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 -- 172.21.15.72:0/3053870175 shutdown_connections 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/3053870175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c106450 0x7f1b9c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/3053870175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101d30 0x7f1b9c105d70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/3053870175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c101380 0x7f1b9c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 -- 172.21.15.72:0/3053870175 >> 172.21.15.72:0/3053870175 conn(0x7f1b9c078580 msgr2=0x7f1b9c078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:13.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 -- 172.21.15.72:0/3053870175 shutdown_connections 2024-03-11T19:33:13.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.939+0000 7f1ba1e00640 1 -- 172.21.15.72:0/3053870175 wait complete. 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.940+0000 7f1ba1e00640 1 Processor -- start 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.940+0000 7f1ba1e00640 1 -- start start 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c106450 0x7f1b9c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 0x7f1b9c1a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1a5fa0 con 0x7f1b9c101380 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1a6110 con 0x7f1b9c073f00 2024-03-11T19:33:13.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1ba1e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1a62d0 con 0x7f1b9c106450 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49412/0 (socket says 172.21.15.72:49412) 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9aa00640 1 -- 172.21.15.72:0/549458484 learned_addr learned my addr 172.21.15.72:0/549458484 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b92000640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c106450 0x7f1b9c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 0x7f1b9c1a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 -- 172.21.15.72:0/549458484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c106450 msgr2=0x7f1b9c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c106450 0x7f1b9c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 -- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 msgr2=0x7f1b9c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9b400640 1 -- 172.21.15.72:0/549458484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84009560 con 0x7f1b9c073f00 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.941+0000 7f1b9aa00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1b9b400640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 0x7f1b9c1a59d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009fd0 tx=0x7f1b9400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400ce20 con 0x7f1b9c073f00 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b94005020 con 0x7f1b9c073f00 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c1a65b0 con 0x7f1b9c073f00 2024-03-11T19:33:13.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94010690 con 0x7f1b9c073f00 2024-03-11T19:33:13.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.942+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c1a6a40 con 0x7f1b9c073f00 2024-03-11T19:33:13.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.943+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c0683c0 con 0x7f1b9c073f00 2024-03-11T19:33:13.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.945+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1b94005500 con 0x7f1b9c073f00 2024-03-11T19:33:13.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.947+0000 7f1ba0a00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 0x7f1b800a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:13.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.947+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f1b94014070 con 0x7f1b9c073f00 2024-03-11T19:33:13.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.947+0000 7f1b9aa00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 0x7f1b800a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:13.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.948+0000 7f1b9aa00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 0x7f1b800a4090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c011550 tx=0x7f1b8c005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:13.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:13.949+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b94096d40 con 0x7f1b9c073f00 2024-03-11T19:33:14.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.148+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b9c0749b0 con 0x7f1b9c073f00 2024-03-11T19:33:14.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.149+0000 7f1ba0a00640 1 -- 172.21.15.72:0/549458484 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b94005190 con 0x7f1b9c073f00 2024-03-11T19:33:14.153 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 msgr2=0x7f1b800a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 0x7f1b800a4090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c011550 tx=0x7f1b8c005f90 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 msgr2=0x7f1b9c1a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 0x7f1b9c1a59d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009fd0 tx=0x7f1b9400ef10 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 shutdown_connections 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f1b800a1bd0 0x7f1b800a4090 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c073f00 0x7f1b9c1a59d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c106450 0x7f1b9c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 --2- 172.21.15.72:0/549458484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b9c101380 0x7f1b9c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 >> 172.21.15.72:0/549458484 conn(0x7f1b9c078580 msgr2=0x7f1b9c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:14.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.151+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 shutdown_connections 2024-03-11T19:33:14.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.152+0000 7f1ba1e00640 1 -- 172.21.15.72:0/549458484 wait complete. 2024-03-11T19:33:14.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 -- 172.21.15.72:0/127292129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28108660 msgr2=0x7efe28110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 --2- 172.21.15.72:0/127292129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28108660 0x7efe28110230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe24009cd0 tx=0x7efe2402c0e0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 -- 172.21.15.72:0/127292129 shutdown_connections 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 --2- 172.21.15.72:0/127292129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28108660 0x7efe28110230 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 --2- 172.21.15.72:0/127292129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28103f40 0x7efe28107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 --2- 172.21.15.72:0/127292129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe28103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 -- 172.21.15.72:0/127292129 >> 172.21.15.72:0/127292129 conn(0x7efe280fee30 msgr2=0x7efe28101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 -- 172.21.15.72:0/127292129 shutdown_connections 2024-03-11T19:33:14.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.383+0000 7efe30800640 1 -- 172.21.15.72:0/127292129 wait complete. 2024-03-11T19:33:14.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.384+0000 7efe30800640 1 Processor -- start 2024-03-11T19:33:14.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 -- start start 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe2819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28108660 0x7efe2819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819a2e0 con 0x7efe28199830 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819a450 con 0x7efe28108660 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.385+0000 7efe30800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819a5c0 con 0x7efe28103590 2024-03-11T19:33:14.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe2819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49436/0 (socket says 172.21.15.72:49436) 2024-03-11T19:33:14.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 -- 172.21.15.72:0/1363063908 learned_addr learned my addr 172.21.15.72:0/1363063908 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:14.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2d400640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28108660 0x7efe2819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 -- 172.21.15.72:0/1363063908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 msgr2=0x7efe2819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe2819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 -- 172.21.15.72:0/1363063908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28108660 msgr2=0x7efe2819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28108660 0x7efe2819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 -- 172.21.15.72:0/1363063908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c009490 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2de00640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe2819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.386+0000 7efe2e800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe24004af0 tx=0x7efe24034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.387+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240328c0 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.387+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe24003e50 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.387+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24004860 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.387+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe24009900 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.387+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe281a67b0 con 0x7efe28199830 2024-03-11T19:33:14.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.388+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28104a60 con 0x7efe28199830 2024-03-11T19:33:14.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.390+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7efe2400a040 con 0x7efe28199830 2024-03-11T19:33:14.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.392+0000 7efe22a00640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 0x7efe100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.392+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7efe240f0420 con 0x7efe28199830 2024-03-11T19:33:14.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.392+0000 7efe2de00640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 0x7efe100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.393+0000 7efe2de00640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 0x7efe100a4060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011630 tx=0x7efe1c005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:14.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.397+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe240b90e0 con 0x7efe28199830 2024-03-11T19:33:14.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.593+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe2810b930 con 0x7efe28199830 2024-03-11T19:33:14.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.594+0000 7efe22a00640 1 -- 172.21.15.72:0/1363063908 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe240bc7a0 con 0x7efe28199830 2024-03-11T19:33:14.598 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:33:14.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 msgr2=0x7efe100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 0x7efe100a4060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011630 tx=0x7efe1c005f90 comp rx=0 tx=0).stop 2024-03-11T19:33:14.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 msgr2=0x7efe28199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe24004af0 tx=0x7efe24034cc0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 shutdown_connections 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.596+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efe100a1ba0 0x7efe100a4060 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe28199830 0x7efe28199ce0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe28108660 0x7efe2819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 --2- 172.21.15.72:0/1363063908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28103590 0x7efe2819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 >> 172.21.15.72:0/1363063908 conn(0x7efe280fee30 msgr2=0x7efe28101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 shutdown_connections 2024-03-11T19:33:14.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.597+0000 7efe30800640 1 -- 172.21.15.72:0/1363063908 wait complete. 2024-03-11T19:33:14.621 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 -- 172.21.15.72:0/1601679933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f410b7c0 msgr2=0x7f57f410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 --2- 172.21.15.72:0/1601679933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f410b7c0 0x7f57f410dc30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009ca0 tx=0x7f57ec02c190 comp rx=0 tx=0).stop 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 -- 172.21.15.72:0/1601679933 shutdown_connections 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 --2- 172.21.15.72:0/1601679933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f410b7c0 0x7f57f410dc30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 --2- 172.21.15.72:0/1601679933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 --2- 172.21.15.72:0/1601679933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f41023f0 0x7f57f41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 -- 172.21.15.72:0/1601679933 >> 172.21.15.72:0/1601679933 conn(0x7f57f40771e0 msgr2=0x7f57f40775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 -- 172.21.15.72:0/1601679933 shutdown_connections 2024-03-11T19:33:14.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.834+0000 7f57fa800640 1 -- 172.21.15.72:0/1601679933 wait complete. 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.835+0000 7f57fa800640 1 Processor -- start 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 -- start start 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 0x7f57f419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 0x7f57f4197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4197c60 con 0x7f57f41023f0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4197dd0 con 0x7f57f410b7c0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57fa800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4197f90 con 0x7f57f4102d10 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 0x7f57f419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 0x7f57f419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49450/0 (socket says 172.21.15.72:49450) 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57f3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 0x7f57f4197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.836+0000 7f57f3400640 1 -- 172.21.15.72:0/583537911 learned_addr learned my addr 172.21.15.72:0/583537911 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 -- 172.21.15.72:0/583537911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 msgr2=0x7f57f419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 -- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 msgr2=0x7f57f419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 0x7f57f419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 -- 172.21.15.72:0/583537911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009900 con 0x7f57f410b7c0 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57eaa00640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:14.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f3e00640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 0x7f57f4197610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c7a0 tx=0x7f57ec0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:14.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003d80 con 0x7f57f410b7c0 2024-03-11T19:33:14.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ec0348e0 con 0x7f57f410b7c0 2024-03-11T19:33:14.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f4198210 con 0x7f57f410b7c0 2024-03-11T19:33:14.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec004540 con 0x7f57f410b7c0 2024-03-11T19:33:14.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.837+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f41a44b0 con 0x7f57f410b7c0 2024-03-11T19:33:14.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.839+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f40683c0 con 0x7f57f410b7c0 2024-03-11T19:33:14.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.840+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f57ec03b050 con 0x7f57f410b7c0 2024-03-11T19:33:14.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.842+0000 7f57f0c00640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 0x7f57d80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:14.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.842+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f57ec0ef9e0 con 0x7f57f410b7c0 2024-03-11T19:33:14.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.842+0000 7f57f3400640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 0x7f57d80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:14.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.843+0000 7f57f3400640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 0x7f57d80a40e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009c90 tx=0x7f57dc005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:14.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:14.846+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f57ec0b8690 con 0x7f57f410b7c0 2024-03-11T19:33:14.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: from='client.25685 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: pgmap v313: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:33:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2776499692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:33:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/549458484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:33:14.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:14 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1363063908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:33:14.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: from='client.25685 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: pgmap v313: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:33:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2776499692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:33:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/549458484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:33:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:14 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1363063908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: from='client.25685 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: pgmap v313: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2776499692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/549458484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:33:14.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:14 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1363063908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.048+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57f41068b0 con 0x7f57f410b7c0 2024-03-11T19:33:15.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.049+0000 7f57f0c00640 1 -- 172.21.15.72:0/583537911 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57ec0bbd50 con 0x7f57f410b7c0 2024-03-11T19:33:15.053 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:33:15.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 msgr2=0x7f57d80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 0x7f57d80a40e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009c90 tx=0x7f57dc005f90 comp rx=0 tx=0).stop 2024-03-11T19:33:15.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 msgr2=0x7f57f4197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 0x7f57f4197610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c7a0 tx=0x7f57ec0346f0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 shutdown_connections 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f57d80a1c20 0x7f57d80a40e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57f410b7c0 0x7f57f4197610 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57f4102d10 0x7f57f419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 --2- 172.21.15.72:0/583537911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41023f0 0x7f57f419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.051+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 >> 172.21.15.72:0/583537911 conn(0x7f57f40771e0 msgr2=0x7f57f40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.052+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 shutdown_connections 2024-03-11T19:33:15.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.052+0000 7f57fa800640 1 -- 172.21.15.72:0/583537911 wait complete. 2024-03-11T19:33:15.075 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 -- 172.21.15.72:0/3942271197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694101380 msgr2=0x7fa694101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 --2- 172.21.15.72:0/3942271197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694101380 0x7fa694101760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa68402bc30 comp rx=0 tx=0).stop 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 -- 172.21.15.72:0/3942271197 shutdown_connections 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 --2- 172.21.15.72:0/3942271197 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694106450 0x7fa69410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 --2- 172.21.15.72:0/3942271197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa694101d30 0x7fa694105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 --2- 172.21.15.72:0/3942271197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694101380 0x7fa694101760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 -- 172.21.15.72:0/3942271197 >> 172.21.15.72:0/3942271197 conn(0x7fa694078580 msgr2=0x7fa694078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:15.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 -- 172.21.15.72:0/3942271197 shutdown_connections 2024-03-11T19:33:15.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.282+0000 7fa699800640 1 -- 172.21.15.72:0/3942271197 wait complete. 2024-03-11T19:33:15.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.283+0000 7fa699800640 1 Processor -- start 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 -- start start 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 0x7fa69419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6941997c0 0x7fa694199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419a270 con 0x7fa694101380 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419a3e0 con 0x7fa6941997c0 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa699800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419a550 con 0x7fa694106450 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa692a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa692a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49460/0 (socket says 172.21.15.72:49460) 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.284+0000 7fa692a00640 1 -- 172.21.15.72:0/4224020933 learned_addr learned my addr 172.21.15.72:0/4224020933 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 0x7fa69419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa693400640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6941997c0 0x7fa694199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 -- 172.21.15.72:0/4224020933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6941997c0 msgr2=0x7fa694199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6941997c0 0x7fa694199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 -- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 msgr2=0x7fa69419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 -- 172.21.15.72:0/4224020933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684009560 con 0x7fa694106450 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa692a00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.285+0000 7fa68be00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 0x7fa69419fb90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e9c0 tx=0x7fa67c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.286+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00ccc0 con 0x7fa694106450 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.286+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa67c00ce20 con 0x7fa694106450 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.286+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6941a6500 con 0x7fa694106450 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.286+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c018880 con 0x7fa694106450 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.286+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6941a6a10 con 0x7fa694106450 2024-03-11T19:33:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.287+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694102850 con 0x7fa694106450 2024-03-11T19:33:15.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.289+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa67c0189e0 con 0x7fa694106450 2024-03-11T19:33:15.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.290+0000 7fa68b400640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 0x7fa6780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.290+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fa67c014070 con 0x7fa694106450 2024-03-11T19:33:15.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.291+0000 7fa692a00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 0x7fa6780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.291+0000 7fa692a00640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 0x7fa6780a40e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa6840058d0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:15.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.293+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa67c096e90 con 0x7fa694106450 2024-03-11T19:33:15.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.490+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa694109680 con 0x7fa694106450 2024-03-11T19:33:15.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.491+0000 7fa68b400640 1 -- 172.21.15.72:0/4224020933 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa67c0260a0 con 0x7fa694106450 2024-03-11T19:33:15.496 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 msgr2=0x7fa6780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 0x7fa6780a40e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa6840058d0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 msgr2=0x7fa69419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 0x7fa69419fb90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e9c0 tx=0x7fa67c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 shutdown_connections 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa6780a1c20 0x7fa6780a40e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6941997c0 0x7fa694199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa694106450 0x7fa69419fb90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 --2- 172.21.15.72:0/4224020933 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa694101380 0x7fa69419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 >> 172.21.15.72:0/4224020933 conn(0x7fa694078580 msgr2=0x7fa6940ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 shutdown_connections 2024-03-11T19:33:15.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.494+0000 7fa699800640 1 -- 172.21.15.72:0/4224020933 wait complete. 2024-03-11T19:33:15.518 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-11T19:33:15.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 -- 172.21.15.72:0/894358731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b54107730 msgr2=0x7f9b54107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 --2- 172.21.15.72:0/894358731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b54107730 0x7f9b54107bb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009890 tx=0x7f9b4002bb70 comp rx=0 tx=0).stop 2024-03-11T19:33:15.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 -- 172.21.15.72:0/894358731 shutdown_connections 2024-03-11T19:33:15.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 --2- 172.21.15.72:0/894358731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b541080f0 0x7f9b54114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 --2- 172.21.15.72:0/894358731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b54107730 0x7f9b54107bb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 --2- 172.21.15.72:0/894358731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 0x7f9b5410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 -- 172.21.15.72:0/894358731 >> 172.21.15.72:0/894358731 conn(0x7f9b54075710 msgr2=0x7f9b54077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 -- 172.21.15.72:0/894358731 shutdown_connections 2024-03-11T19:33:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.727+0000 7f9b59000640 1 -- 172.21.15.72:0/894358731 wait complete. 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.728+0000 7f9b59000640 1 Processor -- start 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.728+0000 7f9b59000640 1 -- start start 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 0x7f9b5419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 0x7f9b541a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541a2cf0 con 0x7f9b54107730 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541a2e60 con 0x7f9b5410d740 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b59000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541a3020 con 0x7f9b541080f0 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b51e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b51e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49478/0 (socket says 172.21.15.72:49478) 2024-03-11T19:33:15.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b51400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 0x7f9b5419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b51e00640 1 -- 172.21.15.72:0/1205949771 learned_addr learned my addr 172.21.15.72:0/1205949771 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.729+0000 7f9b52800640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 0x7f9b541a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 -- 172.21.15.72:0/1205949771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 msgr2=0x7f9b541a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 0x7f9b541a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 -- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 msgr2=0x7f9b5419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 -- 172.21.15.72:0/1205949771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009560 con 0x7f9b541080f0 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51e00640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b51400640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 0x7f9b5419e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b40037040 tx=0x7f9b400045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:15.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40034990 con 0x7f9b541080f0 2024-03-11T19:33:15.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b4002c9f0 con 0x7f9b541080f0 2024-03-11T19:33:15.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40032e60 con 0x7f9b541080f0 2024-03-11T19:33:15.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b40009490 con 0x7f9b541080f0 2024-03-11T19:33:15.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.730+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b541a3480 con 0x7f9b541080f0 2024-03-11T19:33:15.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.731+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b54068430 con 0x7f9b541080f0 2024-03-11T19:33:15.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.734+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f9b4003b040 con 0x7f9b541080f0 2024-03-11T19:33:15.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.735+0000 7f9b4aa00640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 0x7f9b380a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:15.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.735+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f9b400ef520 con 0x7f9b541080f0 2024-03-11T19:33:15.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.735+0000 7f9b51e00640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 0x7f9b380a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:15.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.736+0000 7f9b51e00640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 0x7f9b380a40e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b440049d0 tx=0x7f9b44009500 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:15.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.739+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b400b81d0 con 0x7f9b541080f0 2024-03-11T19:33:15.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:15 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/583537911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:15 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/4224020933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:15 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/583537911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:15 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/4224020933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:15 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/583537911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:15 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/4224020933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:33:15.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.988+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f9b5410fd30 con 0x7f9b541080f0 2024-03-11T19:33:16.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:15.998+0000 7f9b4aa00640 1 -- 172.21.15.72:0/1205949771 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+735001 (secure 0 0 0) 0x7f9b5410fd30 con 0x7f9b541080f0 2024-03-11T19:33:16.015 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:33:16.015 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "epoch": 28, 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "active_gid": 15303, 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "active_name": "x", 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addrs": { 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6800", 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3931879529 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-03-11T19:33:16.016 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6801", 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3931879529 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addr": "172.21.15.72:6801/3931879529", 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "active_change": "2024-03-11T19:23:09.242068+0000", 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "active_mgr_features": 4540701547738038271, 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "available": true, 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "standbys": [ 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 25394, 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "y", 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-03-11T19:33:16.017 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.018 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.019 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.020 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.022 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.024 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-03-11T19:33:16.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.027 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.029 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.030 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.031 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-03-11T19:33:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.036 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:33:16.038 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.039 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.040 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.041 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.042 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.043 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.044 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.045 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-03-11T19:33:16.046 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.047 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.048 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.050 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-03-11T19:33:16.051 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-03-11T19:33:16.052 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-03-11T19:33:16.054 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.056 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.058 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.060 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-03-11T19:33:16.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-03-11T19:33:16.062 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:33:16.063 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-03-11T19:33:16.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-03-11T19:33:16.065 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-03-11T19:33:16.066 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-03-11T19:33:16.067 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-03-11T19:33:16.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-03-11T19:33:16.069 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-03-11T19:33:16.070 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-03-11T19:33:16.071 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-03-11T19:33:16.072 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.074 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.075 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.077 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-03-11T19:33:16.078 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.080 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.081 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.082 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:33:16.083 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:33:16.084 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.086 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.087 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.089 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.090 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:33:16.091 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.092 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.094 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.095 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.097 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.098 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.099 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.100 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:33:16.101 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.102 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:33:16.103 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.104 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.107 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-03-11T19:33:16.108 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-03-11T19:33:16.109 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.110 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.111 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.112 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.113 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.114 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.115 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.116 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.117 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.118 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.119 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.120 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.121 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.122 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.123 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.124 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.125 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.126 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.127 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.128 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.130 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.131 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.133 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.134 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.135 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:33:16.136 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.137 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.138 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.139 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.140 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.141 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.142 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.143 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.144 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.146 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.148 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.150 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.151 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.152 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.154 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.155 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:33:16.156 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:33:16.157 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.158 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.160 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.161 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.164 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.168 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.169 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-03-11T19:33:16.170 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.171 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.172 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.173 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.174 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:16.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.176 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.177 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.178 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.179 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.180 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.181 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.182 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.183 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.184 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.185 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.186 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.187 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.188 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.189 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.190 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.191 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.192 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.194 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.195 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.196 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.197 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.198 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.199 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:33:16.200 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.201 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.202 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.203 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.204 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.205 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.206 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.207 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.208 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-03-11T19:33:16.209 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:16.210 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-03-11T19:33:16.211 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-03-11T19:33:16.212 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.213 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.214 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.215 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.216 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.217 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.218 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.219 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.220 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.221 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.222 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.223 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.225 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.226 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.227 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.228 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.230 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.231 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-03-11T19:33:16.232 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.233 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.234 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.236 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.237 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.238 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.239 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:33:16.241 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:33:16.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.243 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.245 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.246 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.247 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.248 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.249 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.250 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.251 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.252 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.253 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.254 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.255 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.256 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.257 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.258 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-03-11T19:33:16.259 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.260 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.261 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.263 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.264 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.265 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.267 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.268 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.269 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.270 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.271 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.272 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.273 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.274 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.275 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.276 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.277 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.278 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.279 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.280 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.281 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.282 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.283 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.284 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.285 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.286 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.287 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.288 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.289 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.290 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-03-11T19:33:16.291 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.292 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.293 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-03-11T19:33:16.294 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.295 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.296 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.297 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.298 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.299 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-03-11T19:33:16.300 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:33:16.301 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.302 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.303 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-03-11T19:33:16.304 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.305 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.306 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.307 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.308 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.309 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.310 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.311 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.312 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:33:16.313 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:33:16.314 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-03-11T19:33:16.315 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.316 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.317 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.318 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.319 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.320 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.321 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.322 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.323 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.324 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.325 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.326 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.327 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.328 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.329 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.330 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.331 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.332 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.333 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.334 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.335 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.336 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.337 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.338 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.339 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.340 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.341 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.343 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.344 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.345 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.346 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.347 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.348 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.349 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.350 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.351 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.352 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.353 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.354 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.355 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.356 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.357 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.358 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.359 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.360 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.361 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.362 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.363 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.364 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.365 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.366 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.367 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.369 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.370 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.371 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.372 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.374 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.375 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.376 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.377 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.378 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.379 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.380 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.381 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.382 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:33:16.383 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.384 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.385 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.386 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.388 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.390 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.391 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.392 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.393 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.394 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.395 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.396 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.397 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.398 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.399 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.400 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.401 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.402 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.403 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.404 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.405 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.406 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.407 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.408 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.409 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.410 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.411 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.412 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.413 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.414 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.415 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.416 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.417 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.418 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.419 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.420 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.422 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.423 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 25402, 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "z", 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-03-11T19:33:16.424 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.425 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.426 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.427 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.428 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.429 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.430 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-03-11T19:33:16.431 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-03-11T19:33:16.432 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-03-11T19:33:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.435 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.438 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.439 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.440 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-03-11T19:33:16.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.442 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-03-11T19:33:16.443 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-03-11T19:33:16.444 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-03-11T19:33:16.445 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.446 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.447 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.448 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.449 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.450 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.451 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-03-11T19:33:16.452 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.453 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.454 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.455 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:33:16.456 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.457 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.458 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-03-11T19:33:16.459 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.460 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.461 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.462 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.463 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.464 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.465 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.466 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.467 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.468 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.469 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.470 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.471 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:33:16.472 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.473 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:33:16.474 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.475 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.476 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-03-11T19:33:16.477 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-03-11T19:33:16.478 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.479 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-03-11T19:33:16.480 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:33:16.481 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.482 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.483 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.485 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.486 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.487 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.488 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.489 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.490 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.491 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.492 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-03-11T19:33:16.493 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.494 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.495 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.496 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.497 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:33:16.498 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.499 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.500 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.501 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.502 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:33:16.503 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:33:16.504 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.505 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.506 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.507 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.508 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:33:16.509 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.510 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.511 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.512 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.513 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.514 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.515 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.516 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.517 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.518 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.519 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.520 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.521 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.522 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.523 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.524 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:33:16.525 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:33:16.526 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:33:16.527 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:33:16.528 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.529 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.530 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.531 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.532 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.533 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.534 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.535 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.536 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.537 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.538 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.539 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.540 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:33:16.541 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:33:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:33:16.543 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.544 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.545 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:33:16.546 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.547 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.548 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.549 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.550 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.551 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.552 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.553 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.554 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.555 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.556 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.557 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.558 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.559 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:33:16.560 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.561 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:33:16.562 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:33:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.564 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.565 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.566 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.567 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.568 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.569 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.570 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.571 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.572 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.573 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.574 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.575 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.576 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.577 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.578 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.579 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-03-11T19:33:16.580 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:16.581 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.582 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.583 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.584 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.585 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.586 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.587 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.588 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.589 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.590 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.591 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.592 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.593 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-03-11T19:33:16.594 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.595 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.596 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.597 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.598 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.599 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.600 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.601 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.602 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.603 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.604 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.605 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:33:16.606 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:33:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:33:16.608 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.609 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.610 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.611 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.612 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.613 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-03-11T19:33:16.614 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-03-11T19:33:16.615 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.616 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.617 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.618 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.619 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.620 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.621 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.622 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.623 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.624 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.625 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.626 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.627 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.629 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.630 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.631 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.632 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.633 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.634 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.635 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.636 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-03-11T19:33:16.638 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.639 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.640 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.641 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.642 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.643 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.644 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-03-11T19:33:16.645 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.647 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.648 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.649 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.650 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.651 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.652 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.653 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.654 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.655 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.656 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.657 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.658 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.659 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.660 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.661 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.662 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.663 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.664 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.665 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.666 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.667 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.668 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.669 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.670 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.671 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.672 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.673 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.674 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.675 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.676 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.677 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.678 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.679 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.680 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.681 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.682 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.683 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.684 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.685 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.686 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.687 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.688 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.689 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.690 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.691 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.692 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.693 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.696 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.697 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.698 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.699 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.700 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.701 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.702 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.703 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.704 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.705 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.706 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.708 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.709 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.710 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.711 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.712 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-03-11T19:33:16.713 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.715 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.716 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.717 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.718 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.719 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.720 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.721 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.722 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.723 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-03-11T19:33:16.724 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.725 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:16.726 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.727 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.728 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.729 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.730 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.731 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.732 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.733 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.734 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.735 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.736 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.737 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.738 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.739 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.740 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.741 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.742 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.743 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.744 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.745 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.746 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.747 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.748 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.749 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.750 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.751 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.752 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-03-11T19:33:16.753 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-03-11T19:33:16.754 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-03-11T19:33:16.755 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-03-11T19:33:16.756 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-03-11T19:33:16.757 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-03-11T19:33:16.758 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-03-11T19:33:16.759 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-03-11T19:33:16.760 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-03-11T19:33:16.761 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-03-11T19:33:16.762 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.763 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-03-11T19:33:16.764 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.765 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:33:16.766 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.767 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.768 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.769 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.770 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.771 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.772 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.773 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.774 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.775 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.776 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.777 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.778 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.779 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.780 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.781 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.782 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-03-11T19:33:16.783 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.784 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.785 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.786 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.787 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.788 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.789 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.790 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.791 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.792 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.794 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.796 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-03-11T19:33:16.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.799 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.801 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.802 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.803 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.804 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.805 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.806 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.807 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.809 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.810 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.811 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.812 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.813 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.814 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.815 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-03-11T19:33:16.816 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.817 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.818 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.819 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.820 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.821 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.822 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.823 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.824 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.825 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.827 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.828 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.829 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.830 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.831 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-03-11T19:33:16.832 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-03-11T19:33:16.833 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.834 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.835 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "modules": [ 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "iostat", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "nfs", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "restful", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "snap_schedule" 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.836 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.837 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.838 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.839 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.840 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.841 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.842 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.843 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.844 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.845 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.846 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-03-11T19:33:16.847 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.848 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.849 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.850 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-03-11T19:33:16.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-03-11T19:33:16.852 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.854 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.856 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-03-11T19:33:16.857 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-03-11T19:33:16.858 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.859 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.860 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.862 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.863 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.864 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-03-11T19:33:16.865 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.866 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.868 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.869 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-03-11T19:33:16.870 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.871 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.872 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.874 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.875 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.876 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.877 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.878 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.880 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.881 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.882 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.883 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-03-11T19:33:16.884 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.887 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-03-11T19:33:16.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-03-11T19:33:16.889 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.890 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-03-11T19:33:16.891 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.892 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.893 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.894 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.895 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-03-11T19:33:16.897 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-03-11T19:33:16.898 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.900 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.901 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.902 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:33:16.903 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.904 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.905 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.907 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.908 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:33:16.909 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.910 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.911 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.912 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.913 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.914 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.915 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-03-11T19:33:16.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.918 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.920 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.921 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.922 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.923 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.924 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-03-11T19:33:16.925 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.926 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:33:16.929 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:33:16.930 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.931 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.932 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.933 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.934 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.935 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.936 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.937 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.942 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.944 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.945 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.947 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:33:16.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.949 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.950 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.951 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.952 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.953 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.954 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.955 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.959 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:33:16.960 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:33:16.961 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:33:16.962 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.963 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.965 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.966 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.968 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.970 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.974 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.976 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:33:16.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.978 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.981 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.982 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.983 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.984 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.985 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-03-11T19:33:16.986 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.987 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.988 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.989 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.990 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:16.991 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.992 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-03-11T19:33:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:33:16.994 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:16.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.996 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.998 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-03-11T19:33:16.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.000 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.001 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.002 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.003 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.004 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.005 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.006 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.007 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.008 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.009 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.010 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:17.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.013 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.014 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.015 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.016 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.017 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.018 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-03-11T19:33:17.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-03-11T19:33:17.023 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.025 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.026 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.027 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.028 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.030 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.031 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.032 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.033 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.034 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.036 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.038 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.039 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.040 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.041 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.042 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.043 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.045 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.046 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.047 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.048 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:33:17.050 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.051 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.052 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.054 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.055 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.056 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.057 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.058 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.059 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.060 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-03-11T19:33:17.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.062 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.063 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.065 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.066 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.067 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.068 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.069 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.071 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.072 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.073 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.074 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.075 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.076 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.077 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.078 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.079 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.080 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.081 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:33:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.085 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.086 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-03-11T19:33:17.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.088 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.089 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.090 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.091 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.092 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.093 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.094 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.095 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.096 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.097 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.098 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.099 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.100 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.101 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.102 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.103 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.104 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.105 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.106 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.107 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.108 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.109 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.110 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.112 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.113 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.114 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.115 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.116 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.117 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.118 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.119 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.120 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.121 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.122 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.123 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.124 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.125 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.126 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.127 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.128 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:33:17.130 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.131 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-03-11T19:33:17.132 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.133 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.134 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.135 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.136 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.138 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.139 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.140 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.141 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.142 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.143 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.144 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.145 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.146 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.147 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.148 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.149 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.150 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.151 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.152 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.153 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.154 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.155 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.156 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.157 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.158 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.159 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.160 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.161 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.162 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.163 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.164 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.165 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.166 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.167 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.168 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.169 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.170 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.171 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.172 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.174 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.175 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.176 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.177 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.178 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.179 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.180 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.181 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.182 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.183 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.185 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.186 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.187 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.188 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.189 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.190 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.191 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.193 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.194 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.195 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.196 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.197 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.198 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.203 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-03-11T19:33:17.204 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-03-11T19:33:17.205 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.206 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.207 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.210 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.211 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.212 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.213 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.214 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.215 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.217 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.218 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.219 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-03-11T19:33:17.221 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.222 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.223 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.224 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.225 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.226 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.227 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.228 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.230 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.231 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.232 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.233 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.234 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.238 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.239 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.241 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.242 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-03-11T19:33:17.243 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.244 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-03-11T19:33:17.247 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.248 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "services": { 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard": "https://172.21.15.72:8443/" 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "always_on_modules": { 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "octopus": [ 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-03-11T19:33:17.249 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "pacific": [ 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "quincy": [ 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-03-11T19:33:17.250 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "reef": [ 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-03-11T19:33:17.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "squid": [ 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-03-11T19:33:17.252 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "last_failure_osd_epoch": 95, 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "active_clients": [ 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2274733445 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "libcephsqlite", 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 1951240272 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-03-11T19:33:17.254 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3694001208 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2451965211 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.255 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 3942179699 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-03-11T19:33:17.256 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 678073339 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-03-11T19:33:17.257 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:33:17.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:16 smithi143 ceph-mon[36013]: pgmap v314: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 10 op/s 2024-03-11T19:33:17.259 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:16 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1205949771' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:33:17.259 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:16 smithi106 ceph-mon[35889]: pgmap v314: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 10 op/s 2024-03-11T19:33:17.259 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:16 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1205949771' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:33:17.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 msgr2=0x7f9b380a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:17.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 0x7f9b380a40e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b440049d0 tx=0x7f9b44009500 comp rx=0 tx=0).stop 2024-03-11T19:33:17.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 msgr2=0x7f9b5419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:17.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 0x7f9b5419e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b40037040 tx=0x7f9b400045b0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 shutdown_connections 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f9b380a1c20 0x7f9b380a40e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b5410d740 0x7f9b541a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b541080f0 0x7f9b5419e370 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 --2- 172.21.15.72:0/1205949771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b54107730 0x7f9b5419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 >> 172.21.15.72:0/1205949771 conn(0x7f9b54075710 msgr2=0x7f9b54075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 shutdown_connections 2024-03-11T19:33:17.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:33:16.008+0000 7f9b59000640 1 -- 172.21.15.72:0/1205949771 wait complete. 2024-03-11T19:33:17.266 DEBUG:tasks.check_counter:Getting stats from a 2024-03-11T19:33:17.266 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-11T19:33:17.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:16 smithi072 ceph-mon[31695]: pgmap v314: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 10 op/s 2024-03-11T19:33:17.267 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:16 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1205949771' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:33:17.713 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:17.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:17 smithi072 ceph-mon[31695]: pgmap v315: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:17.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:17 smithi143 ceph-mon[36013]: pgmap v315: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:18.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:17 smithi106 ceph-mon[35889]: pgmap v315: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:20.435 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.004220474, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.001003704, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:20.436 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 6, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 172, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3392, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 185296, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.035770704, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.024949443, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.004739916, 2024-03-11T19:33:20.437 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 170, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.027837173, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000163748 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 185296 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.438 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 234, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 221, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1839408, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 132896, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.108992874, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.035102058, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.090588182, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.011149270, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 220, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.020404934, 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000092749 2024-03-11T19:33:20.439 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1839408, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 132896 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:20.440 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:20.441 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:20.442 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:20.443 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148714, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 30, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:20.444 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:20.445 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 236, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1806443, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 236, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 236, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1806443, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:20.446 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:20.447 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:20.447 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:20.447 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:20.447 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:33:20.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:20 smithi106 ceph-mon[35889]: pgmap v316: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:20.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:20 smithi072 ceph-mon[31695]: pgmap v316: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:20.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:20 smithi143 ceph-mon[36013]: pgmap v316: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-11T19:33:21.327 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-11T19:33:21.328 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-11T19:33:21.328 DEBUG:tasks.check_counter:Getting stats from d 2024-03-11T19:33:21.328 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-11T19:33:21.776 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:22.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:22 smithi106 ceph-mon[35889]: pgmap v317: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-11T19:33:22.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:22 smithi072 ceph-mon[31695]: pgmap v317: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-11T19:33:22.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:22 smithi143 ceph-mon[36013]: pgmap v317: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-11T19:33:24.834 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:24 smithi072 ceph-mon[31695]: pgmap v318: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 7 op/s 2024-03-11T19:33:24.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:24 smithi106 ceph-mon[35889]: pgmap v318: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 7 op/s 2024-03-11T19:33:24.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:24 smithi143 ceph-mon[36013]: pgmap v318: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 7 op/s 2024-03-11T19:33:25.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:25 smithi072 ceph-mon[31695]: pgmap v319: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-11T19:33:25.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:25 smithi143 ceph-mon[36013]: pgmap v319: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-11T19:33:26.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:25 smithi106 ceph-mon[35889]: pgmap v319: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-11T19:33:27.862 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 4, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 4, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3072, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 448, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.003633393, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000061004, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000974164, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.001311809, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000437269 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 242, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 229, 2024-03-11T19:33:27.864 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1841584, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 137888, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.114756470, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.024732947, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.124024363, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.010634092, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 228, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.021435310, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000094014 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:27.865 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1841584, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 137888 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 2, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 176, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 185904, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.035757383, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.025677506, 2024-03-11T19:33:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.004060573, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 173, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.030404685, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000175749 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 185904 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.867 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:27.868 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:27.869 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:27.870 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148714, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 30, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-03-11T19:33:27.871 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:27.872 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 244, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1807483, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 244, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 244, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1807483, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:27.873 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:33:28.680 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:28 smithi072 ceph-mon[31695]: pgmap v320: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:28.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:28 smithi106 ceph-mon[35889]: pgmap v320: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-11T19:33:28.924 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-11T19:33:28.924 DEBUG:tasks.check_counter:Getting stats from g 2024-03-11T19:33:28.924 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-11T19:33:28.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:28 smithi143 ceph-mon[36013]: pgmap v320: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:29.372 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:29.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:29 smithi072 ceph-mon[31695]: pgmap v321: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:29.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:29 smithi143 ceph-mon[36013]: pgmap v321: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:30.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:29 smithi106 ceph-mon[35889]: pgmap v321: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 406, 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 549, 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 135025, 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 328266, 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 9, 2024-03-11T19:33:31.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.123801467, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.055150028, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.056930057, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.007761455, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 547, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.083019215, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000151771 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 892, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000460606, 2024-03-11T19:33:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000516 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 5152 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 820, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 545, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 2070904, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 238909, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 12, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.241448570, 2024-03-11T19:33:31.903 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.083512759, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.210593693, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.012166643, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 545, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.054887596, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000100711 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 772, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000240899, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000312 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1912416, 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 148544 2024-03-11T19:33:31.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 23, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 196, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 6043, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 471438, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 10, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.043369294, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.027167781, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.008978860, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000351809, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 193, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.047136230, 2024-03-11T19:33:31.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000244229 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 22, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000004810, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000218 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 461584 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-03-11T19:33:31.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 51, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002721470, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000053362 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000007683, 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000003841 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 5, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 3, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000921711, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000307237 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 2, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 11, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-03-11T19:33:31.908 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 19, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 2, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 17, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 13, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 9, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 4, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 19, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 11, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 2, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 4, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:31.909 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-03-11T19:33:31.910 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:31.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 15, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 9, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 11, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:31.912 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:31.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 8, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 8, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 2, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 2, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 2, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-03-11T19:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 4196657, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 0, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.025356744, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.003622392 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 0 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 24, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 30, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 6, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 23, 2024-03-11T19:33:31.915 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 28, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 5, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 19, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 28, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 9, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 2, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 2, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 50944, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 207228 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 6, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 1, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 5, 2024-03-11T19:33:31.916 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 24, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 2, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000326748, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000326748 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000594963, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000297481 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.921 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-03-11T19:33:31.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.927 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 2, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 4, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 2, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 2, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 2, 2024-03-11T19:33:31.928 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 1, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 600, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:31.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:31.930 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 264327, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 94, 2024-03-11T19:33:31.931 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 201, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:31.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 100421, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 384, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 46, 2024-03-11T19:33:31.933 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 12890, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 27, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 46, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 46, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.290993692, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.006325949 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 46, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 70 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 46, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 0, 2024-03-11T19:33:31.934 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 46, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 10, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 7, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 8, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 6, 2024-03-11T19:33:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 20, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-03-11T19:33:31.936 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-03-11T19:33:31.937 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 103, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 13, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 9, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 9, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 17, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 2 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 1, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-03-11T19:33:31.938 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:31.939 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 1245, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 2080831, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 1245, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1245, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 2080831, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.940 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 46, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 12890, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 46, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 12890, 2024-03-11T19:33:31.941 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:31.942 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.942 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.942 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.942 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 46, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 46, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 46, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 46, 2024-03-11T19:33:31.943 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:31.944 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x55cde557e6e0": { 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 8, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 2353, 2024-03-11T19:33:31.945 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 8, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 7, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 2353, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:31.946 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: pgmap v322: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:32 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.898 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: pgmap v322: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:32.898 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.899 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:32 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.930 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-11T19:33:32.930 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-11T19:33:32.930 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-11T19:33:32.930 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=15 2024-03-11T19:33:32.930 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-11T19:33:32.931 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-11T19:33:32.931 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T19:33:32.931 DEBUG:tasks.check_counter:Getting stats from j 2024-03-11T19:33:32.931 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-11T19:33:32.939 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: pgmap v322: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:32.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:32 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:33.357 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:34.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:34 smithi106 ceph-mon[35889]: pgmap v323: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:34.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:34 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:34 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:34 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:34 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:34 smithi072 ceph-mon[31695]: pgmap v323: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:34 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:34 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:34 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:34 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:34 smithi143 ceph-mon[36013]: pgmap v323: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:34 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:34 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:34 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:34.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:34 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:35 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.932 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:35 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:35.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:35 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:36.369 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-03-11T19:33:36.369 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:36.369 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 249, 2024-03-11T19:33:36.369 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 234, 2024-03-11T19:33:36.369 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1845472, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 140208, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.113879825, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.036450415, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.091962972, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.011126265, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 234, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.023598386, 2024-03-11T19:33:36.370 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000100847 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1845472, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 140208 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 2, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 178, 2024-03-11T19:33:36.371 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 186256, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.039000424, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.027414859, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.005244401, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 175, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.033213944, 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000189793 2024-03-11T19:33:36.372 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 186256 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:33:36.373 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.000850244, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000726409, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:36.374 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-03-11T19:33:36.375 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:36.376 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:36.377 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148754, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 31, 2024-03-11T19:33:36.378 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-03-11T19:33:36.379 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 247, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1807873, 2024-03-11T19:33:36.380 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 247, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 247, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1807873, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-03-11T19:33:36.381 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-03-11T19:33:36.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:36 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:33:36.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:36 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:36 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:33:36.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:36 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:36 smithi106 ceph-mon[35889]: pgmap v324: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:36.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:36 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:33:36.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:36 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:36 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:33:36.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:36 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:36 smithi072 ceph-mon[31695]: pgmap v324: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:36 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi143 to 174.1M 2024-03-11T19:33:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:36 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi143 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:36 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-11T19:33:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:36 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-11T19:33:36.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:36 smithi143 ceph-mon[36013]: pgmap v324: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-11T19:33:37.309 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-11T19:33:37.309 DEBUG:tasks.check_counter:Getting stats from b 2024-03-11T19:33:37.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-11T19:33:37.559 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:33:37.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:37 smithi072 ceph-mon[31695]: pgmap v325: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:37.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:37 smithi143 ceph-mon[36013]: pgmap v325: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:38.060 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:37 smithi106 ceph-mon[35889]: pgmap v325: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:39.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:39 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:39 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:39 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:39 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:39 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:39 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:39.512 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-11T19:33:39.512 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:39.512 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1019, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 802, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 461464, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 792972, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 27, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 6, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.112959011, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.030307905, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.039108580, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.034878086, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 801, 2024-03-11T19:33:39.513 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.145225523, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000181305 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 967, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000204126, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000211 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 44704, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 4592 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:39.514 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1337, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1021, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 588921, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 813600, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 12, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.126711050, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.039773223, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.057158241, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.031716947, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1019, 2024-03-11T19:33:39.515 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.167445556, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000164323 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1730, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000335955, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000194 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 43024, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 549088 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 2099, 2024-03-11T19:33:39.516 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 861, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 2666073, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 548964, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 24, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 9, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.271526045, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.064986750, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.127780724, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.091096892, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 857, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.171176953, 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000199739 2024-03-11T19:33:39.517 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1796, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000346659, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000193 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 2593360, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 302176 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-11T19:33:39.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 476, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.076295771, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000160285 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 32, 2024-03-11T19:33:39.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001118605, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000034956 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 177, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 177, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 177, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2.614368310, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.014770442 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.520 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 64, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 30, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 7, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 12, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 11, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 20, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 7, 2024-03-11T19:33:39.521 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 2, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 2, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 3, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 237, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 223, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 1, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 1, 2024-03-11T19:33:39.522 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 2, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 1, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 2, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:39.523 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 2, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 20, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 14, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 29, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 4, 2024-03-11T19:33:39.524 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 4, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 23 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 7, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 4, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 2, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 2, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 16, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 1, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 3, 2024-03-11T19:33:39.525 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 2, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 1, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:39.526 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 155, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 95, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 95, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 60, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 3, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 13, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 12, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 12, 2024-03-11T19:33:39.527 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 1, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 1, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 1, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4467078, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 4535215, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 126, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.250660586, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001989369 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-03-11T19:33:39.528 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 26, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 34, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 8, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 21, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 26, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 5, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 30, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 40, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 10, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 2, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 23, 2024-03-11T19:33:39.529 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 21, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 52304, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 215420 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 236, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 4, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 177, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 39, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 13, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6, 2024-03-11T19:33:39.530 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.004532088, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000755348 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 115, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.055093773, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000479076 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-03-11T19:33:39.531 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2.416071321, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.402678553 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.532 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.533 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.002312554, 2024-03-11T19:33:39.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000231255 2024-03-11T19:33:39.535 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.535 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-03-11T19:33:39.535 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.012761852, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001417983 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.057050285, 2024-03-11T19:33:39.536 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005705028 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001194692, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001194692 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-03-11T19:33:39.537 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.002509329, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001254664 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.538 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.016352744, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.008176372 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.021070744, 2024-03-11T19:33:39.539 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005267686 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001276571, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000638285 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:39.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.022974980, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002552775 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001167377, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001167377 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.543 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 7, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 5, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 27, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 13, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 607, 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:39.544 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 120, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 120, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:39.545 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:39.546 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:39.547 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 347660, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 140, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 704, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 8, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 201, 2024-03-11T19:33:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 122327, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 821, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-03-11T19:33:39.549 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 852, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 445101, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 31, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 852, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 852, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 22.400435852, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.026291591 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-03-11T19:33:39.550 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 852, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1039 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 852, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 2, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 850, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 44, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 33, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 2, 2024-03-11T19:33:39.551 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 128, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 62, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 523, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 61, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:39.552 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-03-11T19:33:39.553 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-03-11T19:33:39.554 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 18, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 173, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 13 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.555 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 10, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-03-11T19:33:39.556 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 1, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 11, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 2, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 4451, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 3218706, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:39.557 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 4451, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 4451, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 3218706, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:39.558 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 852, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 445311, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 852, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 445311, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.559 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 852, 2024-03-11T19:33:39.560 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 852, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 852, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 852, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:39.561 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 2, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 210, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 2, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 210, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.562 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x5624dd0a86e0": { 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 155, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 340911, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 155, 2024-03-11T19:33:39.563 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 126, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 340911, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:39.564 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-11T19:33:40.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:33:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:33:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:40 smithi106 ceph-mon[35889]: pgmap v326: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:40.407 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-11T19:33:40.407 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-11T19:33:40.407 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=7 2024-03-11T19:33:40.407 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=4 2024-03-11T19:33:40.408 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-11T19:33:40.408 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-11T19:33:40.408 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-11T19:33:40.408 DEBUG:tasks.check_counter:Getting stats from e 2024-03-11T19:33:40.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:40.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:40 smithi072 ceph-mon[31695]: pgmap v326: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: Adjusting osd_memory_target on smithi072 to 176.7M 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: Unable to set osd_memory_target on smithi072 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' 2024-03-11T19:33:40.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:40 smithi143 ceph-mon[36013]: pgmap v326: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:40.655 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:33:42.491 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 0, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 2, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 0, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.000947994, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000101448, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:42.492 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 249, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 234, 2024-03-11T19:33:42.493 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1845472, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 140208, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.053371285, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.015787354, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.043355555, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.006018224, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 234, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.014297593, 2024-03-11T19:33:42.494 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000061100 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.495 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1845472, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 140208 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 3, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 180, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 480, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 264160, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 6, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:42.496 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.012701915, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.008613177, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.001588100, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 176, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.029101671, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000165350 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.497 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 264160 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 0, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:42.498 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:42.499 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 148670, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 30, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:42.501 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 2, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:42.502 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 248, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 1807900, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 248, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 248, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 1807900, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:42.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:42.504 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:42.504 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:42.504 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-11T19:33:42.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:42 smithi106 ceph-mon[35889]: pgmap v327: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:42.919 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:42 smithi072 ceph-mon[31695]: pgmap v327: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:42.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:42 smithi143 ceph-mon[36013]: pgmap v327: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-03-11T19:33:43.299 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-03-11T19:33:43.299 DEBUG:tasks.check_counter:Getting stats from h 2024-03-11T19:33:43.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-11T19:33:43.541 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:33:44.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:43 smithi106 ceph-mon[35889]: pgmap v328: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:44.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:43 smithi072 ceph-mon[31695]: pgmap v328: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:44.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:43 smithi143 ceph-mon[36013]: pgmap v328: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:45.373 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1698, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1698, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 847082, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 527292, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 6, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.151382526, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.059941288, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.034191389, 2024-03-11T19:33:45.374 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.042726257, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1697, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.063966050, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000037693 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:45.375 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 6, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 180, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1976, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 461422, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.010723003, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.006977514, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.001784943, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000100443, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 177, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.026572384, 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000150126 2024-03-11T19:33:45.376 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 460112 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1079, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1059, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 2093763, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 393703, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 3, 2024-03-11T19:33:45.377 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.118810821, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.034242325, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.057281390, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.032600098, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1058, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.033177405, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000031358 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.378 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1894432, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 141792 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4190, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.108198380, 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000025823 2024-03-11T19:33:45.379 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 0, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 0, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.380 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 21, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 7, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 14, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 0, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 12, 2024-03-11T19:33:45.381 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 2, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 0, 2024-03-11T19:33:45.382 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:45.383 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 0 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 0, 2024-03-11T19:33:45.384 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-03-11T19:33:45.385 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 155, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-03-11T19:33:45.386 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 2, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 13, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4467078, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 4535215, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 4535079, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 155 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 16, 2024-03-11T19:33:45.387 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 20, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 4, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 11, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 13, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 2, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 21, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 24, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 3, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 0, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 0, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 0, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 47820, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 215420 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 0, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 0, 2024-03-11T19:33:45.388 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.389 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.390 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.391 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.392 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.393 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-03-11T19:33:45.394 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.395 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.396 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.397 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.398 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 7, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 5, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 4, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-03-11T19:33:45.399 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 7684, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:45.400 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:45.401 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 190250, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 55, 2024-03-11T19:33:45.402 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 201, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 73844, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 432, 2024-03-11T19:33:45.403 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 2508, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 26, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 4, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 2508, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2508, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.959976012, 2024-03-11T19:33:45.404 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000382765 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2508, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2509 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 2508, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 2508, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 1618, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 887, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-03-11T19:33:45.405 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 2, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-03-11T19:33:45.406 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-03-11T19:33:45.407 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 13, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 5, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 5, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-03-11T19:33:45.408 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 2, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 0, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:45.409 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 2779, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 2675908, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:45.410 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 2779, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2779, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 2675908, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:45.411 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 2508, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 340921, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2508, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 340921, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:45.412 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 2508, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 2508, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2508, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 2508, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.413 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-03-11T19:33:45.414 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x5601fdc10960": { 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:45.415 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:45.416 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-11T19:33:46.204 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-11T19:33:46.204 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-11T19:33:46.205 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-11T19:33:46.205 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-11T19:33:46.205 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-11T19:33:46.205 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-11T19:33:46.205 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T19:33:46.205 DEBUG:tasks.check_counter:Getting stats from k 2024-03-11T19:33:46.205 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-11T19:33:46.445 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:33:46.835 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:46 smithi106 ceph-mon[35889]: pgmap v329: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:46.893 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:46 smithi143 ceph-mon[36013]: pgmap v329: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:46.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:46 smithi072 ceph-mon[31695]: pgmap v329: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:33:47.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:47 smithi106 ceph-mon[35889]: pgmap v330: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:47.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:47 smithi072 ceph-mon[31695]: pgmap v330: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:47.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:47 smithi143 ceph-mon[36013]: pgmap v330: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:48.183 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 249, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 236, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1843488, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 142256, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.055470569, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.015324224, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.045101707, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.006341149, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 235, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.013830435, 2024-03-11T19:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000058852 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1843488, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 142256 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 6, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 183, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 3392, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 187232, 2024-03-11T19:33:48.185 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.012000108, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.007891753, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.001713484, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 181, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.035611358, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000196747 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:48.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 187232 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 0, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 2, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 0, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.000264935, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000169681, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:48.187 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 0, 2024-03-11T19:33:48.188 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:48.189 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:48.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 148714, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 30, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:48.191 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 2, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-11T19:33:48.192 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 251, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 1808393, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 251, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 251, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 1808393, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-11T19:33:48.193 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-11T19:33:48.194 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:48.194 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-11T19:33:48.194 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-11T19:33:48.886 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-11T19:33:48.886 DEBUG:tasks.check_counter:Getting stats from c 2024-03-11T19:33:48.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-11T19:33:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:50.690 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:50 smithi143 ceph-mon[36013]: pgmap v331: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:50.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:50 smithi106 ceph-mon[35889]: pgmap v331: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:50.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:50 smithi072 ceph-mon[31695]: pgmap v331: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1085, 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1065, 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 2168656, 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 396345, 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.151766098, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.047789889, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.064899741, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.037939854, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1064, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.062674091, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000058904 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.016 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1894816, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 142880 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 4, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 181, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 876, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 464524, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 6, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.011599103, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.007596222, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.001841708, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000062653, 2024-03-11T19:33:51.017 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 178, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.036342537, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000204171 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 463840 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.018 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1482, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1482, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 382698, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 460113, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.104985209, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.036216818, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.025254462, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.035734145, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1481, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.053302499, 2024-03-11T19:33:51.019 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000035990 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.020 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3930, 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.113499440, 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000028880 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:51.021 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.022 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-03-11T19:33:51.023 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-03-11T19:33:51.024 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:51.025 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-03-11T19:33:51.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:51.027 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-03-11T19:33:51.028 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 8, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 2, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-03-11T19:33:51.029 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 4196657, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 4196516, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 8 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 3, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 3, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 0, 2024-03-11T19:33:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 1, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 1, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 46856, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215420 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-03-11T19:33:51.031 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:51.032 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.033 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.034 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.036 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-03-11T19:33:51.036 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.036 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.036 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.036 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.037 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:51.038 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:51.039 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.040 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.041 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 4, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 2, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 3, 2024-03-11T19:33:51.043 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 5892, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:51.044 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:51.045 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 182099, 2024-03-11T19:33:51.046 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 49, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 201, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-03-11T19:33:51.047 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 8320, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 13, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 2295, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 2295, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2295, 2024-03-11T19:33:51.048 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.990972214, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000431796 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2295, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2296 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 2295, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 2295, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1468, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-03-11T19:33:51.049 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 825, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:51.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-03-11T19:33:51.051 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 103, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 13, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 4, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 4, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-03-11T19:33:51.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-03-11T19:33:51.053 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 2567, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 2304339, 2024-03-11T19:33:51.054 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 2567, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2567, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2304339, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:51.055 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2295, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2358, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2295, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2358, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.056 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2295, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2295, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2295, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2295, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.057 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-03-11T19:33:51.058 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x5653fab56960": { 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:51.059 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:51.060 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-11T19:33:51.860 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T19:33:51.860 DEBUG:tasks.check_counter:Getting stats from f 2024-03-11T19:33:51.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-11T19:33:51.887 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:51 smithi143 ceph-mon[36013]: pgmap v332: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:51.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:51 smithi072 ceph-mon[31695]: pgmap v332: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:52.111 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:52.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:51 smithi106 ceph-mon[35889]: pgmap v332: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:53.863 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:53 smithi143 ceph-mon[36013]: pgmap v333: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:53.863 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-03-11T19:33:53.863 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 836, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 838, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 4405479, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 260060, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.087403513, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.024022897, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.024883001, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.029604315, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 836, 2024-03-11T19:33:53.864 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.039348995, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000047068 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 772, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 949, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 6842797, 2024-03-11T19:33:53.865 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 703315, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.088332273, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.033771687, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.022722577, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.022641311, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 946, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.063920016, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000067568 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:53.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 463840 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1120, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1098, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 6374024, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 407060, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.157518900, 2024-03-11T19:33:53.867 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.047215931, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.071572132, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.039189412, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1098, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.060954575, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000055514 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.868 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1898432, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 144576 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4125, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.123556247, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000029953 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.869 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 1218, 2024-03-11T19:33:53.871 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 1218, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 1218, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:53.872 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:53.873 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:33:53.874 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:53.875 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-03-11T19:33:53.876 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 6073, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 9, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-03-11T19:33:53.877 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 19227483, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 19225395, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 6073 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 12, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 1227, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 1215, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 9, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 20, 2024-03-11T19:33:53.878 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 11, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 1218, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 1231, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 13, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 48096, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215420 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-03-11T19:33:53.879 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.882 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-03-11T19:33:53.883 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.884 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-03-11T19:33:53.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.886 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.887 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.890 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 5, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 3, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 4, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 4153, 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:53.891 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:53.892 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:53.893 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 207172, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 65, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:53.894 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 201, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:53.895 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 905030, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 5124, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 2450, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-03-11T19:33:53.896 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 2450, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2450, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2.131940205, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000870179 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2450, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2451 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 2450, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 2450, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-03-11T19:33:53.897 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1558, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 890, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-03-11T19:33:53.898 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-03-11T19:33:53.899 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 103, 2024-03-11T19:33:53.900 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 13, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 7, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 7, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.901 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:53.902 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 2724, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 17360175, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 2724, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2724, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 17360175, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.903 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2450, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 15033184, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2450, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 15033184, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2450, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2450, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2450, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2450, 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:53.905 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:53.906 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x55f921e4c960": { 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:53.907 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:53.908 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-03-11T19:33:54.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:53 smithi106 ceph-mon[35889]: pgmap v333: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:54.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:53 smithi072 ceph-mon[31695]: pgmap v333: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:54.743 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-11T19:33:54.744 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T19:33:54.744 DEBUG:tasks.check_counter:Getting stats from i 2024-03-11T19:33:54.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-11T19:33:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:56.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:55 smithi106 ceph-mon[35889]: pgmap v334: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:56.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:55 smithi072 ceph-mon[31695]: pgmap v334: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:56.189 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:55 smithi143 ceph-mon[36013]: pgmap v334: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 3324, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 3353, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 840010, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 5294746, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 17, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.200696268, 2024-03-11T19:33:56.782 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.054404893, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.055237781, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.076489252, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3352, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.168566773, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000050288 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 66, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000011573, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000175 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.783 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 8720, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 13776 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 2778, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 3220, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 686368, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 8189244, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 11, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 7, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.212824661, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.077255895, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.057875504, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.063911838, 2024-03-11T19:33:56.784 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3217, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.156499263, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000048647 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 724, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000136028, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000187 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 469904 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:56.785 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 7273, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 12513, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 4549542, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 9839360, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 6, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.608279220, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.253635431, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.396811616, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.050858897, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 12512, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.292058792, 2024-03-11T19:33:56.786 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000023342 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 11659, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.002041447, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000175 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3878400, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 4555200 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-03-11T19:33:56.787 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4174, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 1.862413320, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000446193 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 5567, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.192430973, 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000034566 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 2433, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 2429, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2429, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2.224921193, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000915982 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 4, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-03-11T19:33:56.789 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 11, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 19, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 1, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 7, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 11, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 16, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 6, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 2, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 3, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 3, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 2461, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 2438, 2024-03-11T19:33:56.790 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 1, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 3, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 1, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 2, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 1, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 2, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 4, 2024-03-11T19:33:56.791 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 234, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 2, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 17, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 3625, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:33:56.792 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 2425, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 1209, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 13, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 1219 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 4, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 9, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 6, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 7, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 4, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 1, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:33:56.793 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 1209, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 1208, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 1, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-03-11T19:33:56.794 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 6073, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 6073, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 9, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 9, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-03-11T19:33:56.795 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 19227483, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 0, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2410, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 8.390588641, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.003481572 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 0 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 22, 2024-03-11T19:33:56.796 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 1233, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 1211, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 20, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 26, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 6, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 19, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 1237, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 1218, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 3, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 1217, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 1214, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 55288, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 207228 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-03-11T19:33:56.797 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 2666, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 2, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 2433, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 30, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 8, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1204, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.709872319, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000589594 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 5, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.001244139, 2024-03-11T19:33:56.798 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000248827 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.001183904, 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000295976 2024-03-11T19:33:56.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000202232, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000101116 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.800 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.801 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.802 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.802 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.802 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-03-11T19:33:56.802 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.005467172, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001822390 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.803 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.002559345, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001279672 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-03-11T19:33:56.804 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.027863380, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.006965845 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.805 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.806 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.807 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1205, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 1.476528702, 2024-03-11T19:33:56.808 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001225335 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 5, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 3, 2024-03-11T19:33:56.809 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 2, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 1849, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 924, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 618, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:56.810 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:56.811 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:56.812 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 316737, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 134, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 440, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 5, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:56.813 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9640, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 201, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 100995, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 596, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-03-11T19:33:56.814 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 7292, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 15285764, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 33, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 7292, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 7292, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 33.222072827, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.004555961 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 7292, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 7389 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.815 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 7292, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 32, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 7260, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 10, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 32, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 2442, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 27, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-03-11T19:33:56.816 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 4706, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 20, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-03-11T19:33:56.817 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-03-11T19:33:56.818 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 103, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 13, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 12, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 12, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 136, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 16 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 1, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:33:56.819 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 19, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 5, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 4691, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 1208, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.820 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 13371, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 4622788, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 13371, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 13371, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 4622788, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.821 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 7292, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 15417420, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 7292, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 15417420, 2024-03-11T19:33:56.822 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 7292, 2024-03-11T19:33:56.823 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 7292, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 7292, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 7292, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 1208, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 131656, 2024-03-11T19:33:56.824 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 1208, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 32, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 131656, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x561964a766e0": { 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-03-11T19:33:56.825 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 6073, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 15033179, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 6073, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2410, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 15033179, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.826 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:56.827 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-03-11T19:33:57.619 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-11T19:33:57.619 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-03-11T19:33:57.619 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=4 2024-03-11T19:33:57.619 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=9 2024-03-11T19:33:57.619 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-03-11T19:33:57.620 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-11T19:33:57.620 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T19:33:57.620 DEBUG:tasks.check_counter:Getting stats from l 2024-03-11T19:33:57.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid d6472314-dfda-11ee-95c8-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-11T19:33:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:57.980 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:57 smithi143 ceph-mon[36013]: pgmap v335: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:58.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:57 smithi106 ceph-mon[35889]: pgmap v335: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:58.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:57 smithi072 ceph-mon[31695]: pgmap v335: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 4, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 4, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 3072, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 448, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 0, 2024-03-11T19:33:59.603 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.001307077, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000040779, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.000528680, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000672202, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000224067 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:59.604 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 2, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 184, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 187312, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.011306606, 2024-03-11T19:33:59.605 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.008008656, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.000994605, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 181, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.024506577, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000135395 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:59.606 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 187312 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 254, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 241, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1845472, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 145088, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 2, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.055295508, 2024-03-11T19:33:59.607 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.016207379, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.043413098, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.006180070, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 240, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.014081087, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000058671 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.608 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1845472, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 145088 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 0, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:33:59.609 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-03-11T19:33:59.610 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:33:59.611 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 148692, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 30, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-03-11T19:33:59.612 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-03-11T19:33:59.613 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 928, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 2, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-03-11T19:33:59.614 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 256, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 1809680, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 256, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 256, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1809680, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:59.615 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-03-11T19:33:59.616 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-03-11T19:33:59.975 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:33:59 smithi143 ceph-mon[36013]: pgmap v336: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:34:00.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:33:59 smithi106 ceph-mon[35889]: pgmap v336: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:34:00.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:33:59 smithi072 ceph-mon[31695]: pgmap v336: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-11T19:34:00.344 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-11T19:34:00.345 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-11T19:34:00.356 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-11T19:34:00.356 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:34:00.356 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:34:00.417 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-03-11T19:34:00.417 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:34:00.468 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:34:00.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 -- 172.21.15.72:0/2591636809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 msgr2=0x7fa36c069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:00.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 --2- 172.21.15.72:0/2591636809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c069bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa360009860 tx=0x7fa36002bb70 comp rx=0 tx=0).stop 2024-03-11T19:34:00.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 -- 172.21.15.72:0/2591636809 shutdown_connections 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 --2- 172.21.15.72:0/2591636809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c10db90 0x7fa36c10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 --2- 172.21.15.72:0/2591636809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c06a130 0x7fa36c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 --2- 172.21.15.72:0/2591636809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c069bf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 -- 172.21.15.72:0/2591636809 >> 172.21.15.72:0/2591636809 conn(0x7fa36c077120 msgr2=0x7fa36c077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 -- 172.21.15.72:0/2591636809 shutdown_connections 2024-03-11T19:34:00.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.677+0000 7fa373600640 1 -- 172.21.15.72:0/2591636809 wait complete. 2024-03-11T19:34:00.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.678+0000 7fa373600640 1 Processor -- start 2024-03-11T19:34:00.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 -- start start 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c06a130 0x7fa36c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c10db90 0x7fa36c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c199ee0 con 0x7fa36c10db90 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c19a050 con 0x7fa36c069810 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.679+0000 7fa373600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c19a210 con 0x7fa36c06a130 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa371600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c10db90 0x7fa36c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56376/0 (socket says 172.21.15.72:56376) 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa36be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c06a130 0x7fa36c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 -- 172.21.15.72:0/3266106958 learned_addr learned my addr 172.21.15.72:0/3266106958 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 -- 172.21.15.72:0/3266106958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c06a130 msgr2=0x7fa36c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:00.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c06a130 0x7fa36c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 -- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c10db90 msgr2=0x7fa36c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:00.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c10db90 0x7fa36c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 -- 172.21.15.72:0/3266106958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360009490 con 0x7fa36c069810 2024-03-11T19:34:00.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa370c00640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa360037040 tx=0x7fa3600045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:00.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.680+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600349d0 con 0x7fa36c069810 2024-03-11T19:34:00.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.681+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa36002c9f0 con 0x7fa36c069810 2024-03-11T19:34:00.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.681+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c19a490 con 0x7fa36c069810 2024-03-11T19:34:00.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.681+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600035f0 con 0x7fa36c069810 2024-03-11T19:34:00.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.681+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c1a6700 con 0x7fa36c069810 2024-03-11T19:34:00.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.682+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c19a760 con 0x7fa36c069810 2024-03-11T19:34:00.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.684+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fa36003b040 con 0x7fa36c069810 2024-03-11T19:34:00.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.686+0000 7fa369600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 0x7fa3540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:00.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.686+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fa3600ef6f0 con 0x7fa36c069810 2024-03-11T19:34:00.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.686+0000 7fa36be00640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 0x7fa3540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:00.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.687+0000 7fa36be00640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 0x7fa3540a4060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa36c1015b0 tx=0x7fa358005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:00.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.690+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa36c19a760 con 0x7fa36c069810 2024-03-11T19:34:00.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.914+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa36c19a760 con 0x7fa36c069810 2024-03-11T19:34:00.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.917+0000 7fa369600640 1 -- 172.21.15.72:0/3266106958 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25336 (secure 0 0 0) 0x7fa36c19a760 con 0x7fa36c069810 2024-03-11T19:34:00.921 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:34:00.922 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":103,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:33:11.664414+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/537254334":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1379647566":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2158720129":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/4059778386":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/828846469":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3915050930":"2024-03-12T19:23:09.241200+0000","172.21.15.72:6800/537254334":"2024-03-12T19:23:09.241200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 msgr2=0x7fa3540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 0x7fa3540a4060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa36c1015b0 tx=0x7fa358005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 msgr2=0x7fa36c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa360037040 tx=0x7fa3600045b0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 shutdown_connections 2024-03-11T19:34:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.919+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fa3540a1ba0 0x7fa3540a4060 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa36c10db90 0x7fa36c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa36c06a130 0x7fa36c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 --2- 172.21.15.72:0/3266106958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa36c069810 0x7fa36c19f6d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 >> 172.21.15.72:0/3266106958 conn(0x7fa36c077120 msgr2=0x7fa36c10f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 shutdown_connections 2024-03-11T19:34:00.925 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:00.920+0000 7fa373600640 1 -- 172.21.15.72:0/3266106958 wait complete. 2024-03-11T19:34:00.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:14:20.581262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:19:39.983904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:19:41.338988+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:19:45.291658+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:34:00.946 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 -- 172.21.15.72:0/1040443417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668103f40 msgr2=0x7fb668107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 --2- 172.21.15.72:0/1040443417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668103f40 0x7fb668107f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb650009930 tx=0x7fb65002bc20 comp rx=0 tx=0).stop 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 -- 172.21.15.72:0/1040443417 shutdown_connections 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 --2- 172.21.15.72:0/1040443417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668108660 0x7fb668110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 --2- 172.21.15.72:0/1040443417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668103f40 0x7fb668107f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 --2- 172.21.15.72:0/1040443417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb668103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 -- 172.21.15.72:0/1040443417 >> 172.21.15.72:0/1040443417 conn(0x7fb6680fee30 msgr2=0x7fb668101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:01.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 -- 172.21.15.72:0/1040443417 shutdown_connections 2024-03-11T19:34:01.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.156+0000 7fb66e600640 1 -- 172.21.15.72:0/1040443417 wait complete. 2024-03-11T19:34:01.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.157+0000 7fb66e600640 1 Processor -- start 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.157+0000 7fb66e600640 1 -- start start 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668108660 0x7fb66819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668199870 0x7fb668199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66819a260 con 0x7fb668108660 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66819a3d0 con 0x7fb668103590 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb66e600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66819a590 con 0x7fb668199870 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb65ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668108660 0x7fb66819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:56392/0 (socket says 172.21.15.72:56392) 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 -- 172.21.15.72:0/3853346692 learned_addr learned my addr 172.21.15.72:0/3853346692 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667e00640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668199870 0x7fb668199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 -- 172.21.15.72:0/3853346692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668199870 msgr2=0x7fb668199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668199870 0x7fb668199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 -- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668108660 msgr2=0x7fb66819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668108660 0x7fb66819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.158+0000 7fb667400640 1 -- 172.21.15.72:0/3853346692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb650009560 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb667400640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb65800ebc0 tx=0x7fb65800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580099b0 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb65800eec0 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658018b60 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6681a6580 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.159+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6681a6a60 con 0x7fb668103590 2024-03-11T19:34:01.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.160+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6680683c0 con 0x7fb668103590 2024-03-11T19:34:01.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.162+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb658018cc0 con 0x7fb668103590 2024-03-11T19:34:01.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.164+0000 7fb664c00640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 0x7fb64c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.164+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fb658014070 con 0x7fb668103590 2024-03-11T19:34:01.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.164+0000 7fb65ea00640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 0x7fb64c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.165+0000 7fb65ea00640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 0x7fb64c0a4060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb65002c160 tx=0x7fb650000bc0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:01.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.166+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6580972b0 con 0x7fb668103590 2024-03-11T19:34:01.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.361+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb668103f40 con 0x7fb668103590 2024-03-11T19:34:01.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.362+0000 7fb664c00640 1 -- 172.21.15.72:0/3853346692 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb65809a970 con 0x7fb668103590 2024-03-11T19:34:01.366 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 msgr2=0x7fb64c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 0x7fb64c0a4060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb65002c160 tx=0x7fb650000bc0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 msgr2=0x7fb66819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb65800ebc0 tx=0x7fb65800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 shutdown_connections 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb64c0a1ba0 0x7fb64c0a4060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb668199870 0x7fb668199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb668108660 0x7fb66819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 --2- 172.21.15.72:0/3853346692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb668103590 0x7fb66819f6c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 >> 172.21.15.72:0/3853346692 conn(0x7fb6680fee30 msgr2=0x7fb668100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:01.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 shutdown_connections 2024-03-11T19:34:01.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.364+0000 7fb66e600640 1 -- 172.21.15.72:0/3853346692 wait complete. 2024-03-11T19:34:01.385 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:00 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3266106958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:01.388 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:34:01.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:00 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3266106958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:01.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:00 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3266106958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:01.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.599+0000 7fcd81e00640 1 -- 172.21.15.72:0/4171083685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 msgr2=0x7fcd7c10fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.599+0000 7fcd81e00640 1 --2- 172.21.15.72:0/4171083685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c10fd50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009be0 tx=0x7fcd7402bef0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 -- 172.21.15.72:0/4171083685 shutdown_connections 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 --2- 172.21.15.72:0/4171083685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c10fd50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 --2- 172.21.15.72:0/4171083685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c100760 0x7fcd7c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 --2- 172.21.15.72:0/4171083685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 -- 172.21.15.72:0/4171083685 >> 172.21.15.72:0/4171083685 conn(0x7fcd7c0fb9d0 msgr2=0x7fcd7c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 -- 172.21.15.72:0/4171083685 shutdown_connections 2024-03-11T19:34:01.605 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.600+0000 7fcd81e00640 1 -- 172.21.15.72:0/4171083685 wait complete. 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 Processor -- start 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 -- start start 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c100760 0x7fcd7c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c199de0 con 0x7fcd7c10d960 2024-03-11T19:34:01.607 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c199f50 con 0x7fcd7c100760 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c19a110 con 0x7fcd7c0ffdb0 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.601+0000 7fcd81400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:56864/0 (socket says 172.21.15.72:56864) 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd80a00640 1 -- 172.21.15.72:0/3521030558 learned_addr learned my addr 172.21.15.72:0/3521030558 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd7be00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c100760 0x7fcd7c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 -- 172.21.15.72:0/3521030558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 msgr2=0x7fcd7c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 -- 172.21.15.72:0/3521030558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c100760 msgr2=0x7fcd7c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c100760 0x7fcd7c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 -- 172.21.15.72:0/3521030558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009560 con 0x7fcd7c10d960 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd80a00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd81400640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c199650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402c500 tx=0x7fcd74033610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.602+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74003b70 con 0x7fcd7c10d960 2024-03-11T19:34:01.608 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.603+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd7402ce40 con 0x7fcd7c10d960 2024-03-11T19:34:01.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.603+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd74009840 con 0x7fcd7c10d960 2024-03-11T19:34:01.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.603+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7403d850 con 0x7fcd7c10d960 2024-03-11T19:34:01.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.603+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c1a6330 con 0x7fcd7c10d960 2024-03-11T19:34:01.609 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.604+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c068b40 con 0x7fcd7c10d960 2024-03-11T19:34:01.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.606+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fcd74003cd0 con 0x7fcd7c10d960 2024-03-11T19:34:01.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.607+0000 7fcd79600640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 0x7fcd640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:01.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.607+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fcd740f0350 con 0x7fcd7c10d960 2024-03-11T19:34:01.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.607+0000 7fcd80a00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 0x7fcd640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:01.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.608+0000 7fcd80a00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 0x7fcd640a4060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd70011550 tx=0x7fcd70005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:01.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.610+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd740b8fa0 con 0x7fcd7c10d960 2024-03-11T19:34:01.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.807+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd7c108af0 con 0x7fcd7c10d960 2024-03-11T19:34:01.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.808+0000 7fcd79600640 1 -- 172.21.15.72:0/3521030558 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd740bc660 con 0x7fcd7c10d960 2024-03-11T19:34:01.813 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:34:01.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 msgr2=0x7fcd640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 0x7fcd640a4060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd70011550 tx=0x7fcd70005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:01.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 msgr2=0x7fcd7c199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:01.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c199650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd7402c500 tx=0x7fcd74033610 comp rx=0 tx=0).stop 2024-03-11T19:34:01.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 shutdown_connections 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fcd640a1ba0 0x7fcd640a4060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd7c10d960 0x7fcd7c199650 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c100760 0x7fcd7c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 --2- 172.21.15.72:0/3521030558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd7c0ffdb0 0x7fcd7c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 >> 172.21.15.72:0/3521030558 conn(0x7fcd7c0fb9d0 msgr2=0x7fcd7c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 shutdown_connections 2024-03-11T19:34:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:01.811+0000 7fcd81e00640 1 -- 172.21.15.72:0/3521030558 wait complete. 2024-03-11T19:34:01.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:34:02.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.040+0000 7f16a6c00640 1 -- 172.21.15.72:0/634818071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 msgr2=0x7f16a0110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.040+0000 7f16a6c00640 1 --2- 172.21.15.72:0/634818071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a0110230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1698009890 tx=0x7f169802bb70 comp rx=0 tx=0).stop 2024-03-11T19:34:02.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.040+0000 7f16a6c00640 1 -- 172.21.15.72:0/634818071 shutdown_connections 2024-03-11T19:34:02.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.040+0000 7f16a6c00640 1 --2- 172.21.15.72:0/634818071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a0110230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.041+0000 7f16a6c00640 1 --2- 172.21.15.72:0/634818071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a0103f40 0x7f16a0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.041+0000 7f16a6c00640 1 --2- 172.21.15.72:0/634818071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.041+0000 7f16a6c00640 1 -- 172.21.15.72:0/634818071 >> 172.21.15.72:0/634818071 conn(0x7f16a00fedd0 msgr2=0x7f16a01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.041+0000 7f16a6c00640 1 -- 172.21.15.72:0/634818071 shutdown_connections 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.041+0000 7f16a6c00640 1 -- 172.21.15.72:0/634818071 wait complete. 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 Processor -- start 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 -- start start 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a01998b0 0x7f16a0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019a2a0 con 0x7f16a0103590 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019a410 con 0x7f16a0108660 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f16a6c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019a5d0 con 0x7f16a01998b0 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.042+0000 7f169fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46080/0 (socket says 172.21.15.72:46080) 2024-03-11T19:34:02.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 -- 172.21.15.72:0/1731663209 learned_addr learned my addr 172.21.15.72:0/1731663209 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f1697400640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f16a4c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a01998b0 0x7f16a0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 -- 172.21.15.72:0/1731663209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a01998b0 msgr2=0x7f16a0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a01998b0 0x7f16a0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 -- 172.21.15.72:0/1731663209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 msgr2=0x7f16a019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 -- 172.21.15.72:0/1731663209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009560 con 0x7f16a0103590 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.043+0000 7f169fe00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9c0 tx=0x7f169000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.044+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000cde0 con 0x7f16a0103590 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.044+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1698009490 con 0x7f16a0103590 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.044+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16900035f0 con 0x7f16a0103590 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.044+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690010650 con 0x7f16a0103590 2024-03-11T19:34:02.050 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.044+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a01a68f0 con 0x7f16a0103590 2024-03-11T19:34:02.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.045+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00683c0 con 0x7f16a0103590 2024-03-11T19:34:02.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.047+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f1690003ad0 con 0x7f16a0103590 2024-03-11T19:34:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.048+0000 7f169d600640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 0x7f16840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.048+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f1690014070 con 0x7f16a0103590 2024-03-11T19:34:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.048+0000 7f1697400640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 0x7f16840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.049+0000 7f1697400640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 0x7f16840a4060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16a0109c60 tx=0x7f1688009500 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.052+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1690096b00 con 0x7f16a0103590 2024-03-11T19:34:02.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:01 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3853346692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:01 smithi072 ceph-mon[31695]: pgmap v337: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:34:02.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:01 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3521030558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:01 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3853346692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:01 smithi143 ceph-mon[36013]: pgmap v337: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:34:02.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:01 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3521030558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.251+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16a010b8f0 con 0x7f16a0103590 2024-03-11T19:34:02.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.252+0000 7f169d600640 1 -- 172.21.15.72:0/1731663209 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1690003760 con 0x7f16a0103590 2024-03-11T19:34:02.256 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-03-11T19:34:02.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 msgr2=0x7f16840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 0x7f16840a4060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16a0109c60 tx=0x7f1688009500 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 msgr2=0x7f16a019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9c0 tx=0x7f169000ee80 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 shutdown_connections 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f16840a1ba0 0x7f16840a4060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16a01998b0 0x7f16a0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a0108660 0x7f16a019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 --2- 172.21.15.72:0/1731663209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16a0103590 0x7f16a019f6b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 >> 172.21.15.72:0/1731663209 conn(0x7f16a00fedd0 msgr2=0x7f16a0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 shutdown_connections 2024-03-11T19:34:02.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.255+0000 7f16a6c00640 1 -- 172.21.15.72:0/1731663209 wait complete. 2024-03-11T19:34:02.278 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:34:02.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:01 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3853346692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:01 smithi106 ceph-mon[35889]: pgmap v337: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-11T19:34:02.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:01 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3521030558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 -- 172.21.15.72:0/1351093106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4108660 msgr2=0x7f5fe4110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 --2- 172.21.15.72:0/1351093106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4108660 0x7f5fe4110230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009930 tx=0x7f5fdc02bc20 comp rx=0 tx=0).stop 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 -- 172.21.15.72:0/1351093106 shutdown_connections 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 --2- 172.21.15.72:0/1351093106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4108660 0x7f5fe4110230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 --2- 172.21.15.72:0/1351093106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4103f40 0x7f5fe4107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 --2- 172.21.15.72:0/1351093106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4103590 0x7f5fe4103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 -- 172.21.15.72:0/1351093106 >> 172.21.15.72:0/1351093106 conn(0x7f5fe40fedf0 msgr2=0x7f5fe4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 -- 172.21.15.72:0/1351093106 shutdown_connections 2024-03-11T19:34:02.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.495+0000 7f5febe00640 1 -- 172.21.15.72:0/1351093106 wait complete. 2024-03-11T19:34:02.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.496+0000 7f5febe00640 1 Processor -- start 2024-03-11T19:34:02.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 -- start start 2024-03-11T19:34:02.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4103590 0x7f5fe419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4199800 0x7f5fe4199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419a2b0 con 0x7f5fe4103590 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419a420 con 0x7f5fe4199800 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5febe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419a590 con 0x7f5fe4108660 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5fe9400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4103590 0x7f5fe419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5fe8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5fe8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.72:44520/0 (socket says 172.21.15.72:44520) 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.497+0000 7f5fe8a00640 1 -- 172.21.15.72:0/875748600 learned_addr learned my addr 172.21.15.72:0/875748600 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe9e00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4199800 0x7f5fe4199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 -- 172.21.15.72:0/875748600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4199800 msgr2=0x7f5fe4199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4199800 0x7f5fe4199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 -- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4103590 msgr2=0x7f5fe419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4103590 0x7f5fe419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 -- 172.21.15.72:0/875748600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009560 con 0x7f5fe4108660 2024-03-11T19:34:02.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe8a00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000eb90 tx=0x7f5fd000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00099b0 con 0x7f5fe4108660 2024-03-11T19:34:02.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd000ee90 con 0x7f5fe4108660 2024-03-11T19:34:02.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.498+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0018b20 con 0x7f5fe4108660 2024-03-11T19:34:02.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.499+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe41a6540 con 0x7f5fe4108660 2024-03-11T19:34:02.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.499+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe41a69d0 con 0x7f5fe4108660 2024-03-11T19:34:02.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.500+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4104a60 con 0x7f5fe4108660 2024-03-11T19:34:02.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.502+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f5fd0018c80 con 0x7f5fe4108660 2024-03-11T19:34:02.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.505+0000 7f5fe2000640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 0x7f5fc80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.505+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f5fd0014070 con 0x7f5fe4108660 2024-03-11T19:34:02.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.505+0000 7f5fe9400640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 0x7f5fc80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.505+0000 7f5fe9400640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 0x7f5fc80a40e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.509+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5fd0052d20 con 0x7f5fe4108660 2024-03-11T19:34:02.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.704+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5fe410b930 con 0x7f5fe4108660 2024-03-11T19:34:02.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.705+0000 7f5fe2000640 1 -- 172.21.15.72:0/875748600 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5fd0096c80 con 0x7f5fe4108660 2024-03-11T19:34:02.710 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-03-11T19:34:02.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 msgr2=0x7f5fc80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 0x7f5fc80a40e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 msgr2=0x7f5fe419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000eb90 tx=0x7f5fd000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 shutdown_connections 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f5fc80a1c20 0x7f5fc80a40e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fe4199800 0x7f5fe4199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe4108660 0x7f5fe419fbd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 --2- 172.21.15.72:0/875748600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fe4103590 0x7f5fe419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 >> 172.21.15.72:0/875748600 conn(0x7f5fe40fedf0 msgr2=0x7f5fe4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 shutdown_connections 2024-03-11T19:34:02.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.708+0000 7f5febe00640 1 -- 172.21.15.72:0/875748600 wait complete. 2024-03-11T19:34:02.733 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 -- 172.21.15.72:0/1324835172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c103f40 msgr2=0x7f763c107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 --2- 172.21.15.72:0/1324835172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c103f40 0x7f763c107f80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7624009930 tx=0x7f762402bc30 comp rx=0 tx=0).stop 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 -- 172.21.15.72:0/1324835172 shutdown_connections 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 --2- 172.21.15.72:0/1324835172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c108660 0x7f763c110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 --2- 172.21.15.72:0/1324835172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c103f40 0x7f763c107f80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 --2- 172.21.15.72:0/1324835172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c103590 0x7f763c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 -- 172.21.15.72:0/1324835172 >> 172.21.15.72:0/1324835172 conn(0x7f763c0fee30 msgr2=0x7f763c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 -- 172.21.15.72:0/1324835172 shutdown_connections 2024-03-11T19:34:02.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.943+0000 7f7642c00640 1 -- 172.21.15.72:0/1324835172 wait complete. 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 Processor -- start 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 -- start start 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c103590 0x7f763c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c108660 0x7f763c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19a340 con 0x7f763c199890 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19a4b0 con 0x7f763c108660 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.944+0000 7f7642c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19a620 con 0x7f763c103590 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f763be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c103590 0x7f763c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46132/0 (socket says 172.21.15.72:46132) 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 -- 172.21.15.72:0/894024553 learned_addr learned my addr 172.21.15.72:0/894024553 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f763b400640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c108660 0x7f763c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 -- 172.21.15.72:0/894024553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c103590 msgr2=0x7f763c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c103590 0x7f763c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 -- 172.21.15.72:0/894024553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c108660 msgr2=0x7f763c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c108660 0x7f763c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 -- 172.21.15.72:0/894024553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624009560 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.945+0000 7f7640c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb10 tx=0x7f763400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.946+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634004b20 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.946+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7634004c80 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.946+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c1a6580 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.946+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634010650 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.946+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c1a6a10 con 0x7f763c199890 2024-03-11T19:34:02.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.947+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c104a60 con 0x7f763c199890 2024-03-11T19:34:02.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.949+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f7634004df0 con 0x7f763c199890 2024-03-11T19:34:02.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.950+0000 7f7638c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 0x7f76200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:02.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.950+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f7634014070 con 0x7f763c199890 2024-03-11T19:34:02.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.951+0000 7f763be00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 0x7f76200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:02.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.951+0000 7f763be00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 0x7f76200a4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f762c009bd0 tx=0x7f762c005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:02.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:02.952+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7634096dc0 con 0x7f763c199890 2024-03-11T19:34:03.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.162+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f763c0683c0 con 0x7f763c199890 2024-03-11T19:34:03.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.163+0000 7f7638c00640 1 -- 172.21.15.72:0/894024553 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v103) v1 ==== 81+0+1000 (secure 0 0 0) 0x7f763409a480 con 0x7f763c199890 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/537254334 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3531203560 2024-03-12T19:10:56.562024+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/618300141 2024-03-12T19:11:29.909764+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3661300568 2024-03-12T19:10:56.562024+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/2769969675 2024-03-12T19:11:29.909764+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2331163134 2024-03-12T19:10:56.562024+0000 2024-03-11T19:34:03.168 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1618271736 2024-03-12T19:11:29.909764+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/1095172224 2024-03-12T19:10:56.562024+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1379647566 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2158720129 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2881891200 2024-03-12T19:11:29.909764+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/4059778386 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/828846469 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3915050930 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/537254334 2024-03-12T19:23:09.241200+0000 2024-03-11T19:34:03.169 INFO:teuthology.orchestra.run.smithi072.stderr:listed 17 entries 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 msgr2=0x7f76200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 0x7f76200a4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f762c009bd0 tx=0x7f762c005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 msgr2=0x7f763c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb10 tx=0x7f763400c2d0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 shutdown_connections 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f76200a1ba0 0x7f76200a4060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f763c199890 0x7f763c199d40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f763c108660 0x7f763c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 --2- 172.21.15.72:0/894024553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f763c103590 0x7f763c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 >> 172.21.15.72:0/894024553 conn(0x7f763c0fee30 msgr2=0x7f763c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 shutdown_connections 2024-03-11T19:34:03.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.166+0000 7f7642c00640 1 -- 172.21.15.72:0/894024553 wait complete. 2024-03-11T19:34:03.189 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-11T19:34:03.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:34:03.190 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-11T19:34:03.264 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:02 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1731663209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.265 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:02 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/875748600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:34:03.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:34:03.314 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-11T19:34:03.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:34:03.379 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-03-11T19:34:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:02 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1731663209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:02 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/875748600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.416 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-11T19:34:03.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:34:03.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T19:34:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:02 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1731663209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:02 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/875748600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:34:03.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:34:03.530 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-03-11T19:34:03.638 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-11T19:34:03.648 INFO:tasks.cephadm:Teardown begin 2024-03-11T19:34:03.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-11T19:34:03.649 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 -- 172.21.15.72:0/2076628337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4101d30 msgr2=0x7efcc4105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 --2- 172.21.15.72:0/2076628337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4101d30 0x7efcc4105d70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efcac009930 tx=0x7efcac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 -- 172.21.15.72:0/2076628337 shutdown_connections 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 --2- 172.21.15.72:0/2076628337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 --2- 172.21.15.72:0/2076628337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4101d30 0x7efcc4105d70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 --2- 172.21.15.72:0/2076628337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 0x7efcc4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 -- 172.21.15.72:0/2076628337 >> 172.21.15.72:0/2076628337 conn(0x7efcc40fcc20 msgr2=0x7efcc40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:03.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 -- 172.21.15.72:0/2076628337 shutdown_connections 2024-03-11T19:34:03.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.833+0000 7efccba00640 1 -- 172.21.15.72:0/2076628337 wait complete. 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 Processor -- start 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 -- start start 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 0x7efcc419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4199910 0x7efcc4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419a300 con 0x7efcc4199910 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419a470 con 0x7efcc4106450 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.834+0000 7efccba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc419a630 con 0x7efcc4101380 2024-03-11T19:34:03.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:49712/0 (socket says 172.21.15.72:49712) 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 -- 172.21.15.72:0/2843571600 learned_addr learned my addr 172.21.15.72:0/2843571600 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcc9a00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4199910 0x7efcc4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcc9000640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 0x7efcc419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 -- 172.21.15.72:0/2843571600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 msgr2=0x7efcc419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 0x7efcc419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 -- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4199910 msgr2=0x7efcc4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4199910 0x7efcc4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 -- 172.21.15.72:0/2843571600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc009840 con 0x7efcc4106450 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.835+0000 7efcbbe00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efcac009230 tx=0x7efcac02c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.836+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac004350 con 0x7efcc4106450 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.836+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac009560 con 0x7efcc4106450 2024-03-11T19:34:03.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.836+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcac0044b0 con 0x7efcc4106450 2024-03-11T19:34:03.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.836+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac003d30 con 0x7efcc4106450 2024-03-11T19:34:03.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.836+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc41a6940 con 0x7efcc4106450 2024-03-11T19:34:03.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.837+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc40683c0 con 0x7efcc4106450 2024-03-11T19:34:03.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.839+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7efcac004df0 con 0x7efcc4106450 2024-03-11T19:34:03.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.840+0000 7efcc2000640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 0x7efca80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:03.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.841+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7efcac0f0290 con 0x7efcc4106450 2024-03-11T19:34:03.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.841+0000 7efcc9000640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 0x7efca80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:03.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.841+0000 7efcc9000640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 0x7efca80a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcc41007f0 tx=0x7efcb4005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:03.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:03.845+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efcac0b8e30 con 0x7efcc4106450 2024-03-11T19:34:04.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.084+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcc419adf0 con 0x7efcc4106450 2024-03-11T19:34:04.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.085+0000 7efcc2000640 1 -- 172.21.15.72:0/2843571600 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11346 (secure 0 0 0) 0x7efcac0bc4f0 con 0x7efcc4106450 2024-03-11T19:34:04.089 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:34:04.089 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:27:44.836024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:34:04.090 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 31 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 msgr2=0x7efca80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 0x7efca80a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcc41007f0 tx=0x7efcb4005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 msgr2=0x7efcc419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efcac009230 tx=0x7efcac02c680 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 shutdown_connections 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7efca80a1ba0 0x7efca80a4060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efcc4199910 0x7efcc4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcc4106450 0x7efcc419fc50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 --2- 172.21.15.72:0/2843571600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc4101380 0x7efcc419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 >> 172.21.15.72:0/2843571600 conn(0x7efcc40fcc20 msgr2=0x7efcc40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 shutdown_connections 2024-03-11T19:34:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.088+0000 7efccba00640 1 -- 172.21.15.72:0/2843571600 wait complete. 2024-03-11T19:34:04.110 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:34:04.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:03 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/894024553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:34:04.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:03 smithi106 ceph-mon[35889]: pgmap v338: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:34:04.191 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:03 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/894024553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:34:04.192 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:03 smithi072 ceph-mon[31695]: pgmap v338: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 -- 172.21.15.72:0/1462973420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc107730 msgr2=0x7fabbc107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 --2- 172.21.15.72:0/1462973420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc107730 0x7fabbc107bb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faba4009930 tx=0x7faba402bc30 comp rx=0 tx=0).stop 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 -- 172.21.15.72:0/1462973420 shutdown_connections 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 --2- 172.21.15.72:0/1462973420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc1080f0 0x7fabbc114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 --2- 172.21.15.72:0/1462973420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc107730 0x7fabbc107bb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 --2- 172.21.15.72:0/1462973420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc10d740 0x7fabbc10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 -- 172.21.15.72:0/1462973420 >> 172.21.15.72:0/1462973420 conn(0x7fabbc0756d0 msgr2=0x7fabbc077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:04.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 -- 172.21.15.72:0/1462973420 shutdown_connections 2024-03-11T19:34:04.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.327+0000 7fabc1400640 1 -- 172.21.15.72:0/1462973420 wait complete. 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.328+0000 7fabc1400640 1 Processor -- start 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.328+0000 7fabc1400640 1 -- start start 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc107730 0x7fabbc19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 0x7fabbc19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 0x7fabbc1a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1a2cb0 con 0x7fabbc1080f0 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1a2e20 con 0x7fabbc107730 2024-03-11T19:34:04.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabc1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1a2fe0 con 0x7fabbc10d740 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 0x7fabbc19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabbb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 0x7fabbc1a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 0x7fabbc19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46170/0 (socket says 172.21.15.72:46170) 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabba800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc107730 0x7fabbc19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.329+0000 7fabb3e00640 1 -- 172.21.15.72:0/1821576664 learned_addr learned my addr 172.21.15.72:0/1821576664 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 -- 172.21.15.72:0/1821576664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc107730 msgr2=0x7fabbc19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc107730 0x7fabbc19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 -- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 msgr2=0x7fabbc19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 0x7fabbc19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 -- 172.21.15.72:0/1821576664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009490 con 0x7fabbc10d740 2024-03-11T19:34:04.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabbb200640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 0x7fabbc1a26b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400ce20 con 0x7fabbc10d740 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba4009560 con 0x7fabbc10d740 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb4005020 con 0x7fabbc10d740 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4010690 con 0x7fabbc10d740 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.330+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc1aa960 con 0x7fabbc10d740 2024-03-11T19:34:04.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.331+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc0683c0 con 0x7fabbc10d740 2024-03-11T19:34:04.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.333+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fabb4005500 con 0x7fabbc10d740 2024-03-11T19:34:04.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.335+0000 7fabb3400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 0x7faba00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.335+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7fabb4014070 con 0x7fabbc10d740 2024-03-11T19:34:04.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.335+0000 7fabba800640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 0x7faba00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.336+0000 7fabba800640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 0x7faba00a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabac009ce0 tx=0x7fabac005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:04.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.337+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabb4096de0 con 0x7fabbc10d740 2024-03-11T19:34:04.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:03 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/894024553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:34:04.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:03 smithi143 ceph-mon[36013]: pgmap v338: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:34:04.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.572+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabbc10fcf0 con 0x7fabbc10d740 2024-03-11T19:34:04.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.574+0000 7fabb3400640 1 -- 172.21.15.72:0/1821576664 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fabb4005190 con 0x7fabbc10d740 2024-03-11T19:34:04.577 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:34:04.577 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1028801800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1028801800},{"type":"v1","addr":"172.21.15.72:6835","nonce":1028801800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/1103846445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":1103846445},{"type":"v1","addr":"172.21.15.72:6841","nonce":1103846445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/4123124925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":4123124925},{"type":"v1","addr":"172.21.15.106:6833","nonce":4123124925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/239358886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":239358886},{"type":"v1","addr":"172.21.15.143:6833","nonce":239358886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1584863054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1584863054},{"type":"v1","addr":"172.21.15.72:6837","nonce":1584863054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/456413378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":456413378},{"type":"v1","addr":"172.21.15.106:6835","nonce":456413378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:19:41.837094+0000","modified":"2024-03-11T19:27:44.836024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2743792791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2743792791},{"type":"v1","addr":"172.21.15.143:6835","nonce":2743792791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6837/902566146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":902566146},{"type":"v1","addr":"172.21.15.143:6837","nonce":902566146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/2982797341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2982797341},{"type":"v1","addr":"172.21.15.106:6837","nonce":2982797341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6839/3455090526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":3455090526},{"type":"v1","addr":"172.21.15.143:6839","nonce":3455090526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.72:6839/3478523060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":3478523060},{"type":"v1","addr":"172.21.15.72:6839","nonce":3478523060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/806155003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":806155003},{"type":"v1","addr":"172.21.15.106:6839","nonce":806155003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:34:04.578 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 31 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.576+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 msgr2=0x7faba00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.576+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 0x7faba00a4060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabac009ce0 tx=0x7fabac005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.576+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 msgr2=0x7fabbc1a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.576+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 0x7fabbc1a26b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb400ef10 comp rx=0 tx=0).stop 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 shutdown_connections 2024-03-11T19:34:04.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7faba00a1ba0 0x7faba00a4060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabbc10d740 0x7fabbc1a26b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1080f0 0x7fabbc19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 --2- 172.21.15.72:0/1821576664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabbc107730 0x7fabbc19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 >> 172.21.15.72:0/1821576664 conn(0x7fabbc0756d0 msgr2=0x7fabbc075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 shutdown_connections 2024-03-11T19:34:04.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.577+0000 7fabc1400640 1 -- 172.21.15.72:0/1821576664 wait complete. 2024-03-11T19:34:04.598 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:34:04.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 -- 172.21.15.72:0/2557340369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 msgr2=0x7f0e98103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/2557340369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e98103980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009930 tx=0x7f0e8802bc20 comp rx=0 tx=0).stop 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 -- 172.21.15.72:0/2557340369 shutdown_connections 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/2557340369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e98108670 0x7f0e98110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/2557340369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98103f50 0x7f0e98107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/2557340369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e98103980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.804+0000 7f0e9ce00640 1 -- 172.21.15.72:0/2557340369 >> 172.21.15.72:0/2557340369 conn(0x7f0e980fee00 msgr2=0x7f0e98101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.805+0000 7f0e9ce00640 1 -- 172.21.15.72:0/2557340369 shutdown_connections 2024-03-11T19:34:04.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.805+0000 7f0e9ce00640 1 -- 172.21.15.72:0/2557340369 wait complete. 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 Processor -- start 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 -- start start 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98108670 0x7f0e9819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 0x7f0e98199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9819a290 con 0x7f0e981035a0 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9819a400 con 0x7f0e981998a0 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e9ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9819a5c0 con 0x7f0e98108670 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e95e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e95e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46178/0 (socket says 172.21.15.72:46178) 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.806+0000 7f0e95e00640 1 -- 172.21.15.72:0/410970195 learned_addr learned my addr 172.21.15.72:0/410970195 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e8d400640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98108670 0x7f0e9819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e96800640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 0x7f0e98199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 -- 172.21.15.72:0/410970195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98108670 msgr2=0x7f0e9819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98108670 0x7f0e9819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 -- 172.21.15.72:0/410970195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 msgr2=0x7f0e98199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 0x7f0e98199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 -- 172.21.15.72:0/410970195 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009560 con 0x7f0e981035a0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e96800640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 0x7f0e98199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.807+0000 7f0e95e00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000c00 tx=0x7f0e88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.808+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8802dea0 con 0x7f0e981035a0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.808+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e88004440 con 0x7f0e981035a0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.808+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e981a65b0 con 0x7f0e981035a0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.808+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88003c90 con 0x7f0e981035a0 2024-03-11T19:34:04.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.808+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981a6ab0 con 0x7f0e981035a0 2024-03-11T19:34:04.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.809+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e980683c0 con 0x7f0e981035a0 2024-03-11T19:34:04.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.811+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f0e88004df0 con 0x7f0e981035a0 2024-03-11T19:34:04.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.812+0000 7f0e8f400640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 0x7f0e7c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:04.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.812+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f0e880f0100 con 0x7f0e981035a0 2024-03-11T19:34:04.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.812+0000 7f0e8d400640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 0x7f0e7c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:04.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.813+0000 7f0e8d400640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 0x7f0e7c0a40e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e98109c90 tx=0x7f0e8000b410 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:04.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:04.816+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e880b8ca0 con 0x7f0e981035a0 2024-03-11T19:34:05.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.018+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e9819add0 con 0x7f0e981035a0 2024-03-11T19:34:05.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.020+0000 7f0e8f400640 1 -- 172.21.15.72:0/410970195 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+25336 (secure 0 0 0) 0x7f0e880dc030 con 0x7f0e981035a0 2024-03-11T19:34:05.025 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-03-11T19:34:05.025 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":103,"fsid":"d6472314-dfda-11ee-95c8-87774f69a715","created":"2024-03-11T19:10:24.349979+0000","modified":"2024-03-11T19:33:11.664414+0000","last_up_change":"2024-03-11T19:18:43.456054+0000","last_in_change":"2024-03-11T19:18:23.081697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:14:20.581262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:19:39.983904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:19:41.338988+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:19:45.291658+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cbac0ed-5153-4a8d-b65d-ac1a30323f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6803","nonce":1634362166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6805","nonce":1634362166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6809","nonce":1634362166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1634362166},{"type":"v1","addr":"172.21.15.72:6807","nonce":1634362166}]},"public_addr":"172.21.15.72:6803/1634362166","cluster_addr":"172.21.15.72:6805/1634362166","heartbeat_back_addr":"172.21.15.72:6809/1634362166","heartbeat_front_addr":"172.21.15.72:6807/1634362166","state":["exists","up"]},{"osd":1,"uuid":"7963ed34-6f58-4b33-bb70-cc1c532602f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6801","nonce":1421287783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6803","nonce":1421287783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6807","nonce":1421287783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1421287783},{"type":"v1","addr":"172.21.15.106:6805","nonce":1421287783}]},"public_addr":"172.21.15.106:6801/1421287783","cluster_addr":"172.21.15.106:6803/1421287783","heartbeat_back_addr":"172.21.15.106:6807/1421287783","heartbeat_front_addr":"172.21.15.106:6805/1421287783","state":["exists","up"]},{"osd":2,"uuid":"843038a5-98bb-4739-8a50-a0ef1fe16d3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6801","nonce":1986223287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6803","nonce":1986223287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6807","nonce":1986223287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":1986223287},{"type":"v1","addr":"172.21.15.143:6805","nonce":1986223287}]},"public_addr":"172.21.15.143:6801/1986223287","cluster_addr":"172.21.15.143:6803/1986223287","heartbeat_back_addr":"172.21.15.143:6807/1986223287","heartbeat_front_addr":"172.21.15.143:6805/1986223287","state":["exists","up"]},{"osd":3,"uuid":"59081b5d-c259-41da-89c9-0e99ed857bed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6811","nonce":3085969755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6813","nonce":3085969755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6817","nonce":3085969755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3085969755},{"type":"v1","addr":"172.21.15.72:6815","nonce":3085969755}]},"public_addr":"172.21.15.72:6811/3085969755","cluster_addr":"172.21.15.72:6813/3085969755","heartbeat_back_addr":"172.21.15.72:6817/3085969755","heartbeat_front_addr":"172.21.15.72:6815/3085969755","state":["exists","up"]},{"osd":4,"uuid":"55843f51-5ecf-4cc1-81bc-6f4fbb4bf32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6809","nonce":1536676428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6811","nonce":1536676428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6815","nonce":1536676428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1536676428},{"type":"v1","addr":"172.21.15.106:6813","nonce":1536676428}]},"public_addr":"172.21.15.106:6809/1536676428","cluster_addr":"172.21.15.106:6811/1536676428","heartbeat_back_addr":"172.21.15.106:6815/1536676428","heartbeat_front_addr":"172.21.15.106:6813/1536676428","state":["exists","up"]},{"osd":5,"uuid":"316bc6e8-8a64-4578-9251-779c72149fbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6809","nonce":1302557280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6811","nonce":1302557280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6815","nonce":1302557280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1302557280},{"type":"v1","addr":"172.21.15.143:6813","nonce":1302557280}]},"public_addr":"172.21.15.143:6809/1302557280","cluster_addr":"172.21.15.143:6811/1302557280","heartbeat_back_addr":"172.21.15.143:6815/1302557280","heartbeat_front_addr":"172.21.15.143:6813/1302557280","state":["exists","up"]},{"osd":6,"uuid":"725a0d6c-5b33-4408-b3e8-6e0e4bc81b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6819","nonce":2847471062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6821","nonce":2847471062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6825","nonce":2847471062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2847471062},{"type":"v1","addr":"172.21.15.72:6823","nonce":2847471062}]},"public_addr":"172.21.15.72:6819/2847471062","cluster_addr":"172.21.15.72:6821/2847471062","heartbeat_back_addr":"172.21.15.72:6825/2847471062","heartbeat_front_addr":"172.21.15.72:6823/2847471062","state":["exists","up"]},{"osd":7,"uuid":"8358cb46-e68c-49bd-9add-390afbd8ef8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6817","nonce":4113260481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6819","nonce":4113260481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6823","nonce":4113260481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4113260481},{"type":"v1","addr":"172.21.15.106:6821","nonce":4113260481}]},"public_addr":"172.21.15.106:6817/4113260481","cluster_addr":"172.21.15.106:6819/4113260481","heartbeat_back_addr":"172.21.15.106:6823/4113260481","heartbeat_front_addr":"172.21.15.106:6821/4113260481","state":["exists","up"]},{"osd":8,"uuid":"4c936701-2ea3-476a-abad-ecaaf2bdc9e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6817","nonce":1425068959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6819","nonce":1425068959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6823","nonce":1425068959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":1425068959},{"type":"v1","addr":"172.21.15.143:6821","nonce":1425068959}]},"public_addr":"172.21.15.143:6817/1425068959","cluster_addr":"172.21.15.143:6819/1425068959","heartbeat_back_addr":"172.21.15.143:6823/1425068959","heartbeat_front_addr":"172.21.15.143:6821/1425068959","state":["exists","up"]},{"osd":9,"uuid":"7124f938-a535-4c94-8959-2931abf1fdd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6827","nonce":98146808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6829","nonce":98146808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6833","nonce":98146808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":98146808},{"type":"v1","addr":"172.21.15.72:6831","nonce":98146808}]},"public_addr":"172.21.15.72:6827/98146808","cluster_addr":"172.21.15.72:6829/98146808","heartbeat_back_addr":"172.21.15.72:6833/98146808","heartbeat_front_addr":"172.21.15.72:6831/98146808","state":["exists","up"]},{"osd":10,"uuid":"c3cca719-676d-4904-b3be-3f42aef54ae0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6825","nonce":3578514696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6827","nonce":3578514696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6831","nonce":3578514696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3578514696},{"type":"v1","addr":"172.21.15.106:6829","nonce":3578514696}]},"public_addr":"172.21.15.106:6825/3578514696","cluster_addr":"172.21.15.106:6827/3578514696","heartbeat_back_addr":"172.21.15.106:6831/3578514696","heartbeat_front_addr":"172.21.15.106:6829/3578514696","state":["exists","up"]},{"osd":11,"uuid":"cbf1e6e6-c003-4c61-a3d0-a1760ed65098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6825","nonce":3564968375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6827","nonce":3564968375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6831","nonce":3564968375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":3564968375},{"type":"v1","addr":"172.21.15.143:6829","nonce":3564968375}]},"public_addr":"172.21.15.143:6825/3564968375","cluster_addr":"172.21.15.143:6827/3564968375","heartbeat_back_addr":"172.21.15.143:6831/3564968375","heartbeat_front_addr":"172.21.15.143:6829/3564968375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:26.938898+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:13:52.529676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:16.095429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:14:49.972434+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:16.727025+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:15:42.001297+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:18.272013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:16:45.904046+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:11.521375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:17:47.647054+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:15.594878+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:18:41.386250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:6801/537254334":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3531203560":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/618300141":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6800/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/3661300568":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6801/2769969675":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/2331163134":"2024-03-12T19:10:56.562024+0000","172.21.15.72:6800/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1618271736":"2024-03-12T19:11:29.909764+0000","172.21.15.72:6801/1095172224":"2024-03-12T19:10:56.562024+0000","172.21.15.72:0/1379647566":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2158720129":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/2881891200":"2024-03-12T19:11:29.909764+0000","172.21.15.72:0/4059778386":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/828846469":"2024-03-12T19:23:09.241200+0000","172.21.15.72:0/3915050930":"2024-03-12T19:23:09.241200+0000","172.21.15.72:6800/537254334":"2024-03-12T19:23:09.241200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:34:05.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 msgr2=0x7f0e7c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 0x7f0e7c0a40e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e98109c90 tx=0x7f0e8000b410 comp rx=0 tx=0).stop 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 msgr2=0x7f0e9819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000c00 tx=0x7f0e88004040 comp rx=0 tx=0).stop 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 shutdown_connections 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f0e7c0a1c20 0x7f0e7c0a40e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e981998a0 0x7f0e98199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e98108670 0x7f0e9819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 --2- 172.21.15.72:0/410970195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e981035a0 0x7f0e9819f6f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 >> 172.21.15.72:0/410970195 conn(0x7f0e980fee00 msgr2=0x7f0e98100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:05.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 shutdown_connections 2024-03-11T19:34:05.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.023+0000 7f0e9ce00640 1 -- 172.21.15.72:0/410970195 wait complete. 2024-03-11T19:34:05.045 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 -- 172.21.15.72:0/2098501330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258106330 msgr2=0x7f725810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 --2- 172.21.15.72:0/2098501330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258106330 0x7f725810df00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7250009890 tx=0x7f725002bb70 comp rx=0 tx=0).stop 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 -- 172.21.15.72:0/2098501330 shutdown_connections 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 --2- 172.21.15.72:0/2098501330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258106330 0x7f725810df00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 --2- 172.21.15.72:0/2098501330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7258101c10 0x7f7258105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 --2- 172.21.15.72:0/2098501330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258101260 0x7f7258101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 -- 172.21.15.72:0/2098501330 >> 172.21.15.72:0/2098501330 conn(0x7f72580fcb20 msgr2=0x7f72580fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 -- 172.21.15.72:0/2098501330 shutdown_connections 2024-03-11T19:34:05.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.249+0000 7f725e800640 1 -- 172.21.15.72:0/2098501330 wait complete. 2024-03-11T19:34:05.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 Processor -- start 2024-03-11T19:34:05.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 -- start start 2024-03-11T19:34:05.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 0x7f725819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 0x7f725819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72581996a0 0x7f7258199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725819a170 con 0x7f7258106330 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725819a2e0 con 0x7f7258101260 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.250+0000 7f725e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725819a4a0 con 0x7f72581996a0 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725ca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 0x7f725819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 0x7f725819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725ca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 0x7f725819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46196/0 (socket says 172.21.15.72:46196) 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725ca00640 1 -- 172.21.15.72:0/675469040 learned_addr learned my addr 172.21.15.72:0/675469040 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725de00640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72581996a0 0x7f7258199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 -- 172.21.15.72:0/675469040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72581996a0 msgr2=0x7f7258199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72581996a0 0x7f7258199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 -- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 msgr2=0x7f725819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 0x7f725819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 -- 172.21.15.72:0/675469040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248009900 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.251+0000 7f725d400640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 0x7f725819f5e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7248009fd0 tx=0x7f724800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.252+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248003970 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.252+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7248004020 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.252+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7250009490 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.252+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800b9d0 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.252+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72581a6790 con 0x7f7258101260 2024-03-11T19:34:05.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.253+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7258102730 con 0x7f7258101260 2024-03-11T19:34:05.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.255+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f724800a040 con 0x7f7258101260 2024-03-11T19:34:05.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.257+0000 7f7256000640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 0x7f723c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.257+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9510+0+0 (secure 0 0 0) 0x7f72480ce230 con 0x7f7258101260 2024-03-11T19:34:05.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.257+0000 7f725ca00640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 0x7f723c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.257+0000 7f725ca00640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 0x7f723c0a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f725819aef0 tx=0x7f7244005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:05.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.259+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7248096cc0 con 0x7f7258101260 2024-03-11T19:34:05.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/2843571600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:04 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1821576664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/2843571600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.430 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:04 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1821576664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/2843571600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:04 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1821576664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.503+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7258067e90 con 0x7f7258101260 2024-03-11T19:34:05.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.517+0000 7f7256000640 1 -- 172.21.15.72:0/675469040 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f724809a380 con 0x7f7258101260 2024-03-11T19:34:05.521 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 msgr2=0x7f723c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 0x7f723c0a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f725819aef0 tx=0x7f7244005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 msgr2=0x7f725819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 0x7f725819f5e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7248009fd0 tx=0x7f724800c880 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 shutdown_connections 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f723c0a1ba0 0x7f723c0a4060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72581996a0 0x7f7258199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7258106330 0x7f725819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 --2- 172.21.15.72:0/675469040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7258101260 0x7f725819f5e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 >> 172.21.15.72:0/675469040 conn(0x7f72580fcb20 msgr2=0x7f72580fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 shutdown_connections 2024-03-11T19:34:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.520+0000 7f725e800640 1 -- 172.21.15.72:0/675469040 wait complete. 2024-03-11T19:34:05.546 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-11T19:34:05.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.740+0000 7fe38aa00640 1 -- 172.21.15.72:0/1936011733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103f40 msgr2=0x7fe384107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.740+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1936011733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103f40 0x7fe384107f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009900 tx=0x7fe36c02bc40 comp rx=0 tx=0).stop 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 -- 172.21.15.72:0/1936011733 shutdown_connections 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1936011733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384108660 0x7fe384110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1936011733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103f40 0x7fe384107f80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1936011733 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384103590 0x7fe384103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 -- 172.21.15.72:0/1936011733 >> 172.21.15.72:0/1936011733 conn(0x7fe3840fedd0 msgr2=0x7fe3841011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 -- 172.21.15.72:0/1936011733 shutdown_connections 2024-03-11T19:34:05.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.741+0000 7fe38aa00640 1 -- 172.21.15.72:0/1936011733 wait complete. 2024-03-11T19:34:05.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 Processor -- start 2024-03-11T19:34:05.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 -- start start 2024-03-11T19:34:05.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 0x7fe38419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384108660 0x7fe38419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 0x7fe384199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38419a2e0 con 0x7fe384108660 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38419a450 con 0x7fe384199830 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.742+0000 7fe38aa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38419a5c0 con 0x7fe384103590 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe388a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 0x7fe384199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe388a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 0x7fe384199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:49798/0 (socket says 172.21.15.72:49798) 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe388a00640 1 -- 172.21.15.72:0/1843798344 learned_addr learned my addr 172.21.15.72:0/1843798344 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 0x7fe38419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383400640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384108660 0x7fe38419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 -- 172.21.15.72:0/1843798344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 msgr2=0x7fe384199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 0x7fe384199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 -- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384108660 msgr2=0x7fe38419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384108660 0x7fe38419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 -- 172.21.15.72:0/1843798344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c009560 con 0x7fe384103590 2024-03-11T19:34:05.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.743+0000 7fe383e00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 0x7fe38419f670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe37400dab0 tx=0x7fe37400df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.744+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400be30 con 0x7fe384103590 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.744+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3841a6520 con 0x7fe384103590 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.744+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe374003e50 con 0x7fe384103590 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.744+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740106c0 con 0x7fe384103590 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.744+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3841a6a00 con 0x7fe384103590 2024-03-11T19:34:05.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.745+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384104a60 con 0x7fe384103590 2024-03-11T19:34:05.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.747+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fe374004000 con 0x7fe384103590 2024-03-11T19:34:05.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.748+0000 7fe380c00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 0x7fe3680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:05.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.748+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9680+0+0 (secure 0 0 0) 0x7fe3740cde10 con 0x7fe384103590 2024-03-11T19:34:05.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.749+0000 7fe383400640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 0x7fe3680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:05.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.749+0000 7fe383400640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 0x7fe3680a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe36c02c250 tx=0x7fe36c005fd0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:05.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.751+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe374096820 con 0x7fe384103590 2024-03-11T19:34:05.896 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:34:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: ignoring --setuser ceph since I am not root 2024-03-11T19:34:05.896 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:34:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: ignoring --setgroup ceph since I am not root 2024-03-11T19:34:05.896 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:34:05 smithi106 ceph-mds[66295]: -- 172.21.15.106:0/1577939454 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56106c62a1a0 con 0x56106c601000 2024-03-11T19:34:05.896 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:34:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: starting mds.b at 2024-03-11T19:34:05.896 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:34:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h[65971]: starting mds.h at 2024-03-11T19:34:05.930 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a[31691]: 2024-03-11T19:34:05.505+0000 7f681e800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:34:05.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:34:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: ignoring --setuser ceph since I am not root 2024-03-11T19:34:05.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:34:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: ignoring --setgroup ceph since I am not root 2024-03-11T19:34:05.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:34:05 smithi072 ceph-mds[73140]: -- 172.21.15.72:0/3494777193 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5557d38c41a0 con 0x5557d389b000 2024-03-11T19:34:05.931 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:34:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: starting mds.g at 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: ignoring --setuser ceph since I am not root 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: ignoring --setgroup ceph since I am not root 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: starting mds.c at 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: ignoring --setuser ceph since I am not root 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: ignoring --setgroup ceph since I am not root 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-mds[66509]: -- 172.21.15.143:0/3519131780 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d2e2be01a0 con 0x55d2e2bb6800 2024-03-11T19:34:05.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: starting mds.f at 2024-03-11T19:34:05.941 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: ignoring --setuser ceph since I am not root 2024-03-11T19:34:05.941 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: ignoring --setgroup ceph since I am not root 2024-03-11T19:34:05.941 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-mds[67122]: -- 172.21.15.143:0/1553984269 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610fef661a0 con 0x5610fef3c800 2024-03-11T19:34:05.941 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:34:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: starting mds.i at 2024-03-11T19:34:05.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:05.987+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3840683c0 con 0x7fe384103590 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/410970195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/675469040' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: pgmap v340: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1843798344' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:06 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/410970195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:06.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/675469040' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.431 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: pgmap v340: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1843798344' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:06 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/410970195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/675469040' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: pgmap v340: 129 pgs: 129 active+clean; 16 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1843798344' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:06 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:34:06.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.522+0000 7fe380c00640 1 -- 172.21.15.72:0/1843798344 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe374099ee0 con 0x7fe384103590 2024-03-11T19:34:06.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 msgr2=0x7fe3680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:06.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 0x7fe3680a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe36c02c250 tx=0x7fe36c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 msgr2=0x7fe38419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:06.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 0x7fe38419f670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe37400dab0 tx=0x7fe37400df70 comp rx=0 tx=0).stop 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 shutdown_connections 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fe3680a1ba0 0x7fe3680a4060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe384199830 0x7fe384199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe384108660 0x7fe38419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 --2- 172.21.15.72:0/1843798344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe384103590 0x7fe38419f670 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.526+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 >> 172.21.15.72:0/1843798344 conn(0x7fe3840fedd0 msgr2=0x7fe384101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:06.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.527+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 shutdown_connections 2024-03-11T19:34:06.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.527+0000 7fe38aa00640 1 -- 172.21.15.72:0/1843798344 wait complete. 2024-03-11T19:34:06.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-11T19:34:06.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 -- 172.21.15.72:0/1171031897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb690107720 msgr2=0x7fb690107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 --2- 172.21.15.72:0/1171031897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb690107720 0x7fb690107ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb678009930 tx=0x7fb67802bc20 comp rx=0 tx=0).stop 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 -- 172.21.15.72:0/1171031897 shutdown_connections 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 --2- 172.21.15.72:0/1171031897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6901080e0 0x7fb6901145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 --2- 172.21.15.72:0/1171031897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb690107720 0x7fb690107ba0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 --2- 172.21.15.72:0/1171031897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb69010d730 0x7fb69010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 -- 172.21.15.72:0/1171031897 >> 172.21.15.72:0/1171031897 conn(0x7fb6900756c0 msgr2=0x7fb690077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 -- 172.21.15.72:0/1171031897 shutdown_connections 2024-03-11T19:34:06.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.757+0000 7fb696200640 1 -- 172.21.15.72:0/1171031897 wait complete. 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.758+0000 7fb696200640 1 Processor -- start 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.758+0000 7fb696200640 1 -- start start 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.758+0000 7fb696200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb690107720 0x7fb69019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb696200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb696200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb69010d730 0x7fb6901a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb696200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a2d10 con 0x7fb6901080e0 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb696200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a2e80 con 0x7fb69010d730 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb696200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901a3040 con 0x7fb690107720 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46236/0 (socket says 172.21.15.72:46236) 2024-03-11T19:34:06.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 -- 172.21.15.72:0/3681303680 learned_addr learned my addr 172.21.15.72:0/3681303680 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb68fe00640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb69010d730 0x7fb6901a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb68f400640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb690107720 0x7fb69019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 -- 172.21.15.72:0/3681303680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb690107720 msgr2=0x7fb69019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb690107720 0x7fb69019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 -- 172.21.15.72:0/3681303680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb69010d730 msgr2=0x7fb6901a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb69010d730 0x7fb6901a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.759+0000 7fb686a00640 1 -- 172.21.15.72:0/3681303680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688009840 con 0x7fb6901080e0 2024-03-11T19:34:06.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb686a00640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb678000c00 tx=0x7fb67802c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67802dea0 con 0x7fb6901080e0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb678009560 con 0x7fb6901080e0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6780042d0 con 0x7fb6901080e0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678003a50 con 0x7fb6901080e0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.760+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6901aa9c0 con 0x7fb6901080e0 2024-03-11T19:34:06.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.761+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6900683c0 con 0x7fb6901080e0 2024-03-11T19:34:06.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.763+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7fb678003bb0 con 0x7fb6901080e0 2024-03-11T19:34:06.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.765+0000 7fb68cc00640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 0x7fb6700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:06.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.765+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9680+0+0 (secure 0 0 0) 0x7fb6780f0100 con 0x7fb6901080e0 2024-03-11T19:34:06.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.765+0000 7fb68f400640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 0x7fb6700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:06.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.765+0000 7fb68f400640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 0x7fb6700a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b50 tx=0x7fb680005f90 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:06.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.769+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6780b8c20 con 0x7fb6901080e0 2024-03-11T19:34:06.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:06.966+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb69019f1a0 con 0x7fb6901080e0 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: Cluster is now healthy 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6835/3763567195,v1:172.21.15.143:6838/3763567195] up:boot 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6836/2985234109,v1:172.21.15.143:6839/2985234109] up:boot 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6836/322530388,v1:172.21.15.106:6838/322530388] up:boot 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.106:6837/2169587447,v1:172.21.15.106:6839/2169587447] up:boot 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.143:6834/2447274421,v1:172.21.15.143:6837/2447274421] up:boot 2024-03-11T19:34:07.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: mds.? [v2:172.21.15.72:6838/3494775308,v1:172.21.15.72:6839/3494775308] up:boot 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: fsmap 12 up:standby 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:34:07.397 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:07 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: Cluster is now healthy 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6835/3763567195,v1:172.21.15.143:6838/3763567195] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6836/2985234109,v1:172.21.15.143:6839/2985234109] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6836/322530388,v1:172.21.15.106:6838/322530388] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.106:6837/2169587447,v1:172.21.15.106:6839/2169587447] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.143:6834/2447274421,v1:172.21.15.143:6837/2447274421] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: mds.? [v2:172.21.15.72:6838/3494775308,v1:172.21.15.72:6839/3494775308] up:boot 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: fsmap 12 up:standby 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:34:07.432 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:34:07.433 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:34:07.433 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:34:07.433 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:34:07.433 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:34:07.433 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:07 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: Cluster is now healthy 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6835/3763567195,v1:172.21.15.143:6838/3763567195] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6836/2985234109,v1:172.21.15.143:6839/2985234109] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6836/322530388,v1:172.21.15.106:6838/322530388] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.106:6837/2169587447,v1:172.21.15.106:6839/2169587447] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.143:6834/2447274421,v1:172.21.15.143:6837/2447274421] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: mds.? [v2:172.21.15.72:6838/3494775308,v1:172.21.15.72:6839/3494775308] up:boot 2024-03-11T19:34:07.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: fsmap 12 up:standby 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='mgr.15303 172.21.15.72:0/283999189' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:34:07.441 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:07 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.522+0000 7fb68cc00640 1 -- 172.21.15.72:0/3681303680 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb678003e60 con 0x7fb6901080e0 2024-03-11T19:34:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2024-03-11T19:34:07.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 msgr2=0x7fb6700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 0x7fb6700a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b50 tx=0x7fb680005f90 comp rx=0 tx=0).stop 2024-03-11T19:34:07.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 msgr2=0x7fb69019e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb678000c00 tx=0x7fb67802c5e0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 shutdown_connections 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7fb6700a1ba0 0x7fb6700a4060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb69010d730 0x7fb6901a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6901080e0 0x7fb69019e390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 --2- 172.21.15.72:0/3681303680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb690107720 0x7fb69019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.526+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 >> 172.21.15.72:0/3681303680 conn(0x7fb6900756c0 msgr2=0x7fb690075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:07.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.527+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 shutdown_connections 2024-03-11T19:34:07.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.527+0000 7fb696200640 1 -- 172.21.15.72:0/3681303680 wait complete. 2024-03-11T19:34:07.554 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-11T19:34:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 -- 172.21.15.72:0/2199375547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 msgr2=0x7f51fc10fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 --2- 172.21.15.72:0/2199375547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc10fc70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009900 tx=0x7f51f402bcd0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 -- 172.21.15.72:0/2199375547 shutdown_connections 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 --2- 172.21.15.72:0/2199375547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc10fc70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 --2- 172.21.15.72:0/2199375547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc073ed0 0x7f51fc10d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 --2- 172.21.15.72:0/2199375547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc0735b0 0x7f51fc073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 -- 172.21.15.72:0/2199375547 >> 172.21.15.72:0/2199375547 conn(0x7f51fc0fcaf0 msgr2=0x7f51fc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 -- 172.21.15.72:0/2199375547 shutdown_connections 2024-03-11T19:34:07.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.753+0000 7f51fbe00640 1 -- 172.21.15.72:0/2199375547 wait complete. 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 Processor -- start 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 -- start start 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 0x7f51fc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc073ed0 0x7f51fc19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc199c70 con 0x7f51fc10d820 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc199de0 con 0x7f51fc0735b0 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.754+0000 7f51fbe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc199fa0 con 0x7f51fc073ed0 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51faa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 0x7f51fc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51faa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 0x7f51fc19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.72:49832/0 (socket says 172.21.15.72:49832) 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51faa00640 1 -- 172.21.15.72:0/1613938284 learned_addr learned my addr 172.21.15.72:0/1613938284 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46262/0 (socket says 172.21.15.72:46262) 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fa000640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc073ed0 0x7f51fc19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 -- 172.21.15.72:0/1613938284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc073ed0 msgr2=0x7f51fc19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc073ed0 0x7f51fc19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 -- 172.21.15.72:0/1613938284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 msgr2=0x7f51fc19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 0x7f51fc19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.755+0000 7f51fb400640 1 -- 172.21.15.72:0/1613938284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4009560 con 0x7f51fc10d820 2024-03-11T19:34:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f51fb400640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009230 tx=0x7f51f402c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40043d0 con 0x7f51fc10d820 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51f40052d0 con 0x7f51fc10d820 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4003c60 con 0x7f51fc10d820 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc19a220 con 0x7f51fc10d820 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.756+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1a6510 con 0x7f51fc10d820 2024-03-11T19:34:07.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.757+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc067e90 con 0x7f51fc10d820 2024-03-11T19:34:07.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.759+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f51f4005440 con 0x7f51fc10d820 2024-03-11T19:34:07.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.760+0000 7f5200a00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 0x7f51e00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:07.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.760+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9184+0+0 (secure 0 0 0) 0x7f51f40f0190 con 0x7f51fc10d820 2024-03-11T19:34:07.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.761+0000 7f51faa00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 0x7f51e00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:07.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.761+0000 7f51faa00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 0x7f51e00a4060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005fd0 tx=0x7f51e400b4a0 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:07.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.764+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51f40b8ea0 con 0x7f51fc10d820 2024-03-11T19:34:07.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:07.962+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51fc1089a0 con 0x7f51fc10d820 2024-03-11T19:34:08.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.534+0000 7f5200a00640 1 -- 172.21.15.72:0/1613938284 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f51f402ce60 con 0x7f51fc10d820 2024-03-11T19:34:08.538 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.538+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 msgr2=0x7f51e00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.538+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 0x7f51e00a4060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005fd0 tx=0x7f51e400b4a0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.538+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 msgr2=0x7f51fc199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.538+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009230 tx=0x7f51f402c7b0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 shutdown_connections 2024-03-11T19:34:08.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f51e00a1ba0 0x7f51e00a4060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51fc10d820 0x7f51fc199620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51fc073ed0 0x7f51fc19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 --2- 172.21.15.72:0/1613938284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc0735b0 0x7f51fc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.539+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 >> 172.21.15.72:0/1613938284 conn(0x7f51fc0fcaf0 msgr2=0x7f51fc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:08.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.540+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 shutdown_connections 2024-03-11T19:34:08.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.541+0000 7f51fbe00640 1 -- 172.21.15.72:0/1613938284 wait complete. 2024-03-11T19:34:08.569 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-11T19:34:08.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.774+0000 7f73cce00640 1 -- 172.21.15.72:0/1822474897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073e40 msgr2=0x7f73c8111ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.774+0000 7f73cce00640 1 --2- 172.21.15.72:0/1822474897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073e40 0x7f73c8111ab0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009900 tx=0x7f73b002bcd0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 -- 172.21.15.72:0/1822474897 shutdown_connections 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 --2- 172.21.15.72:0/1822474897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 0x7f73c81143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 --2- 172.21.15.72:0/1822474897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073e40 0x7f73c8111ab0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 --2- 172.21.15.72:0/1822474897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073520 0x7f73c8073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 -- 172.21.15.72:0/1822474897 >> 172.21.15.72:0/1822474897 conn(0x7f73c80fcaf0 msgr2=0x7f73c80fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 -- 172.21.15.72:0/1822474897 shutdown_connections 2024-03-11T19:34:08.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.775+0000 7f73cce00640 1 -- 172.21.15.72:0/1822474897 wait complete. 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 Processor -- start 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 -- start start 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073e40 0x7f73c81a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 0x7f73c819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c819e170 con 0x7f73c8073520 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c819e2e0 con 0x7f73c8073e40 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73cce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c819e4a0 con 0x7f73c8111ff0 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.776+0000 7f73c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:08.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46278/0 (socket says 172.21.15.72:46278) 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 -- 172.21.15.72:0/919952900 learned_addr learned my addr 172.21.15.72:0/919952900 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73bea00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073e40 0x7f73c81a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7e00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 0x7f73c819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 -- 172.21.15.72:0/919952900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 msgr2=0x7f73c819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 0x7f73c819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 -- 172.21.15.72:0/919952900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073e40 msgr2=0x7f73c81a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073e40 0x7f73c81a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 -- 172.21.15.72:0/919952900 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b0009560 con 0x7f73c8073520 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c7400640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f73b800ebc0 tx=0x7f73b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.777+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b80099b0 con 0x7f73c8073520 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.778+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73b800eec0 con 0x7f73c8073520 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.778+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8018b60 con 0x7f73c8073520 2024-03-11T19:34:08.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.778+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c819e780 con 0x7f73c8073520 2024-03-11T19:34:08.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.778+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81aa9c0 con 0x7f73c8073520 2024-03-11T19:34:08.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.779+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8067e90 con 0x7f73c8073520 2024-03-11T19:34:08.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.781+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 28) v1 ==== 136410+0+0 (secure 0 0 0) 0x7f73b8018cc0 con 0x7f73c8073520 2024-03-11T19:34:08.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.782+0000 7f73c4c00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 0x7f73ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:08.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.782+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8625+0+0 (secure 0 0 0) 0x7f73b8014070 con 0x7f73c8073520 2024-03-11T19:34:08.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.782+0000 7f73bea00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 0x7f73ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:08.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.783+0000 7f73bea00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 0x7f73ac0a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005ab0 tx=0x7f73b0005a20 comp rx=0 tx=0).ready entity=mgr.15303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:08.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.786+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73b8052d20 con 0x7f73c8073520 2024-03-11T19:34:08.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:08.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:08 smithi106 ceph-mon[35889]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T19:34:08.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:08 smithi106 ceph-mon[35889]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:34:08.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:08 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:08.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:08.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:08 smithi072 ceph-mon[31695]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T19:34:08.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:08 smithi072 ceph-mon[31695]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:34:08.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:08 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/3681303680' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:08 smithi143 ceph-mon[36013]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T19:34:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:08 smithi143 ceph-mon[36013]: pgmap v342: 97 pgs: 97 active+clean; 577 KiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:34:08.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:08 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:08.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:08.991+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73c8104460 con 0x7f73c8073520 2024-03-11T19:34:09.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.547+0000 7f73c4c00640 1 -- 172.21.15.72:0/919952900 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) v1 ==== 178+0+0 (secure 0 0 0) 0x7f73b80d1030 con 0x7f73c8073520 2024-03-11T19:34:09.551 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data_ec' removed 2024-03-11T19:34:09.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.553+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 msgr2=0x7f73ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.553+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 0x7f73ac0a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005ab0 tx=0x7f73b0005a20 comp rx=0 tx=0).stop 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.553+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 msgr2=0x7f73c81a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.553+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f73b800ebc0 tx=0x7f73b800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 shutdown_connections 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:6800/3931879529,v1:172.21.15.72:6801/3931879529] conn(0x7f73ac0a1ba0 0x7f73ac0a4060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c8111ff0 0x7f73c819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73c8073e40 0x7f73c81a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:09.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 --2- 172.21.15.72:0/919952900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c8073520 0x7f73c81a3920 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:09.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.554+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 >> 172.21.15.72:0/919952900 conn(0x7f73c80fcaf0 msgr2=0x7f73c80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:09.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.555+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 shutdown_connections 2024-03-11T19:34:09.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-03-11T19:34:09.555+0000 7f73cce00640 1 -- 172.21.15.72:0/919952900 wait complete. 2024-03-11T19:34:09.581 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:34:09.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:34:09.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:34:09.736 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-11T19:34:09.736 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:34:09.792 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:34:09.810 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:34:09.827 INFO:tasks.cephadm:Stopping all daemons... 2024-03-11T19:34:09.827 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-11T19:34:09.827 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a 2024-03-11T19:34:09.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:09 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:09.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:09 smithi106 ceph-mon[35889]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T19:34:09.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:09 smithi106 ceph-mon[35889]: from='client.? 172.21.15.72:0/919952900' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:09.911 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:09 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:09.911 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:09 smithi072 ceph-mon[31695]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T19:34:09.911 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:09 smithi072 ceph-mon[31695]: from='client.? 172.21.15.72:0/919952900' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:09.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:09 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/1613938284' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:34:09.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:09 smithi143 ceph-mon[36013]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T19:34:09.940 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:09 smithi143 ceph-mon[36013]: from='client.? 172.21.15.72:0/919952900' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:34:10.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:09 smithi072 systemd[1]: Stopping Ceph mon.a for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:10.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a[31691]: 2024-03-11T19:34:10.376+0000 7f6825600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:10.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a[31691]: 2024-03-11T19:34:10.376+0000 7f6825600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-11T19:34:10.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:10 smithi072 podman[92558]: 2024-03-11 19:34:10.457995633 +0000 UTC m=+0.221218862 container died d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:34:11.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:10 smithi072 podman[92558]: 2024-03-11 19:34:10.948607689 +0000 UTC m=+0.711830952 container cleanup d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-11T19:34:11.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:10 smithi072 bash[92558]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a 2024-03-11T19:34:11.681 INFO:journalctl@ceph.mon.a.smithi072.stdout:Mar 11 19:34:11 smithi072 podman[92571]: 2024-03-11 19:34:11.209989637 +0000 UTC m=+0.747227865 container remove d1b9521f6196341130e8f416563b8e1e499710c4e788a7b97264745e1f3e8a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-a, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-11T19:34:11.726 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.a.service' 2024-03-11T19:34:11.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:11.849 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-11T19:34:11.849 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-11T19:34:11.849 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.b 2024-03-11T19:34:12.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:11 smithi106 systemd[1]: Stopping Ceph mon.b for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:12.520 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:12 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b[35885]: 2024-03-11T19:34:12.156+0000 7f4bb4663700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:12.520 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:12 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b[35885]: 2024-03-11T19:34:12.156+0000 7f4bb4663700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-11T19:34:12.520 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:12 smithi106 podman[72225]: 2024-03-11 19:34:12.218256013 +0000 UTC m=+0.176997754 container died 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:34:12.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:12 smithi106 podman[72225]: 2024-03-11 19:34:12.62407521 +0000 UTC m=+0.582816957 container cleanup 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:34:12.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:12 smithi106 bash[72225]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b 2024-03-11T19:34:13.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 11 19:34:13 smithi106 podman[72239]: 2024-03-11 19:34:13.017775969 +0000 UTC m=+0.793235871 container remove 57037ee4fb7e06384d47ffa578de5dd48e892a6964ac4f5297df53161815840d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-11T19:34:13.285 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.b.service' 2024-03-11T19:34:13.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:13.334 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-11T19:34:13.334 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-11T19:34:13.334 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.c 2024-03-11T19:34:13.637 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:13 smithi143 systemd[1]: Stopping Ceph mon.c for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:13.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-c[36007]: 2024-03-11T19:34:13.632+0000 7f1b490b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:13.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-c[36007]: 2024-03-11T19:34:13.632+0000 7f1b490b9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-11T19:34:13.941 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:13 smithi143 podman[73176]: 2024-03-11 19:34:13.858427727 +0000 UTC m=+0.331676657 container died 406f788138dfdeb953b2d2242ebfcf3c266ab100960144558eafae772e967c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-11T19:34:14.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:14 smithi143 podman[73176]: 2024-03-11 19:34:14.250698797 +0000 UTC m=+0.723947725 container cleanup 406f788138dfdeb953b2d2242ebfcf3c266ab100960144558eafae772e967c0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-11T19:34:14.440 INFO:journalctl@ceph.mon.c.smithi143.stdout:Mar 11 19:34:14 smithi143 bash[73176]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mon-c 2024-03-11T19:34:14.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mon.c.service' 2024-03-11T19:34:14.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:14.682 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-11T19:34:14.682 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-11T19:34:14.682 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x 2024-03-11T19:34:15.180 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:14 smithi072 systemd[1]: Stopping Ceph mgr.x for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:15.644 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:34:15.247+0000 7fd38b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:15.644 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x[31920]: 2024-03-11T19:34:15.247+0000 7fd38b600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-11T19:34:15.644 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:15 smithi072 podman[92679]: 2024-03-11 19:34:15.340585921 +0000 UTC m=+0.216419760 container died 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-11T19:34:15.930 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:15 smithi072 podman[92679]: 2024-03-11 19:34:15.827511926 +0000 UTC m=+0.703345739 container cleanup 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-11T19:34:15.931 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:15 smithi072 bash[92679]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x 2024-03-11T19:34:16.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Mar 11 19:34:16 smithi072 podman[92692]: 2024-03-11 19:34:16.099705292 +0000 UTC m=+0.752668874 container remove 896de6c503c82ca301f2331729b799110e1b3c4ad78642ba3f5d6457856ae824 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:34:16.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.x.service' 2024-03-11T19:34:16.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:16.744 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-11T19:34:16.744 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-11T19:34:16.745 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.y 2024-03-11T19:34:17.078 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:34:16 smithi106 systemd[1]: Stopping Ceph mgr.y for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:17.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:34:17 smithi106 podman[72345]: 2024-03-11 19:34:17.074534764 +0000 UTC m=+0.139578848 container died 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-11T19:34:17.895 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:34:17 smithi106 podman[72345]: 2024-03-11 19:34:17.629798958 +0000 UTC m=+0.694843030 container cleanup 3596117ded4dbfb81f87a94d9c77b4b2aa8898fd2ae4822ab06c6262b4b55d97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-11T19:34:17.896 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 11 19:34:17 smithi106 bash[72345]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-y 2024-03-11T19:34:18.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.y.service' 2024-03-11T19:34:18.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:18.213 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-11T19:34:18.213 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-11T19:34:18.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z 2024-03-11T19:34:18.592 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:18 smithi143 systemd[1]: Stopping Ceph mgr.z for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:18.843 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:18 smithi143 podman[73299]: 2024-03-11 19:34:18.588808631 +0000 UTC m=+0.170515819 container died 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T19:34:19.190 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 podman[73299]: 2024-03-11 19:34:19.006294212 +0000 UTC m=+0.588001401 container cleanup 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:34:19.190 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 bash[73299]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z 2024-03-11T19:34:19.484 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z.service' 2024-03-11T19:34:19.508 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 podman[73313]: 2024-03-11 19:34:19.241375238 +0000 UTC m=+0.656947556 container remove 90aed54f4f6046a23585dfae743b88af9a9c3167c6502da38059700037a9fa1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-11T19:34:19.508 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 systemd[1]: ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-11T19:34:19.508 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 systemd[1]: ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-11T19:34:19.508 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 systemd[1]: Stopped Ceph mgr.z for d6472314-dfda-11ee-95c8-87774f69a715. 2024-03-11T19:34:19.508 INFO:journalctl@ceph.mgr.z.smithi143.stdout:Mar 11 19:34:19 smithi143 systemd[1]: ceph-d6472314-dfda-11ee-95c8-87774f69a715@mgr.z.service: Consumed 20.907s CPU time. 2024-03-11T19:34:19.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:19.523 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-11T19:34:19.524 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-11T19:34:19.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.0 2024-03-11T19:34:19.930 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:19 smithi072 systemd[1]: Stopping Ceph osd.0 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:20.431 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0[46084]: 2024-03-11T19:34:20.059+0000 7ff52de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:20.431 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0[46084]: 2024-03-11T19:34:20.059+0000 7ff52de00700 -1 osd.0 107 *** Got signal Terminated *** 2024-03-11T19:34:20.431 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0[46084]: 2024-03-11T19:34:20.059+0000 7ff52de00700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:34:25.681 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:25 smithi072 podman[92798]: 2024-03-11 19:34:25.45133992 +0000 UTC m=+5.530623097 container died 99ac7e45d2563ded92ca942743cc6081e14f52b0555990285918382b4756bed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-11T19:34:26.124 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:25 smithi072 podman[92798]: 2024-03-11 19:34:25.8674991 +0000 UTC m=+5.946782313 container cleanup 99ac7e45d2563ded92ca942743cc6081e14f52b0555990285918382b4756bed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-11T19:34:26.124 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:25 smithi072 bash[92798]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0 2024-03-11T19:34:26.430 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:26 smithi072 podman[92817]: 2024-03-11 19:34:26.120247714 +0000 UTC m=+0.655228684 container remove 99ac7e45d2563ded92ca942743cc6081e14f52b0555990285918382b4756bed7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-11T19:34:27.290 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:26 smithi072 podman[92905]: 2024-03-11 19:34:26.837296046 +0000 UTC m=+0.035793947 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:34:27.290 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:26 smithi072 podman[92905]: 2024-03-11 19:34:26.980518256 +0000 UTC m=+0.179016175 container create 7ab2bc210beaac2b908ea8080c4be4a0833b7f3e0156ecd979252d4519496c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True) 2024-03-11T19:34:27.681 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:27 smithi072 podman[92905]: 2024-03-11 19:34:27.285712146 +0000 UTC m=+0.484210119 container init 7ab2bc210beaac2b908ea8080c4be4a0833b7f3e0156ecd979252d4519496c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0-deactivate, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-11T19:34:27.681 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:27 smithi072 podman[92905]: 2024-03-11 19:34:27.296644683 +0000 UTC m=+0.495142608 container start 7ab2bc210beaac2b908ea8080c4be4a0833b7f3e0156ecd979252d4519496c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:34:27.681 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:27 smithi072 podman[92905]: 2024-03-11 19:34:27.368695432 +0000 UTC m=+0.567193337 container attach 7ab2bc210beaac2b908ea8080c4be4a0833b7f3e0156ecd979252d4519496c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True) 2024-03-11T19:34:28.084 INFO:journalctl@ceph.osd.0.smithi072.stdout:Mar 11 19:34:27 smithi072 podman[92905]: 2024-03-11 19:34:27.725350561 +0000 UTC m=+0.923848487 container died 7ab2bc210beaac2b908ea8080c4be4a0833b7f3e0156ecd979252d4519496c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397) 2024-03-11T19:34:28.496 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.0.service' 2024-03-11T19:34:28.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:28.615 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-11T19:34:28.615 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-11T19:34:28.616 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.1 2024-03-11T19:34:28.922 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:28 smithi106 systemd[1]: Stopping Ceph osd.1 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:29.395 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1[40472]: 2024-03-11T19:34:28.916+0000 7f3349b28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:29.396 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1[40472]: 2024-03-11T19:34:28.916+0000 7f3349b28700 -1 osd.1 107 *** Got signal Terminated *** 2024-03-11T19:34:29.396 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1[40472]: 2024-03-11T19:34:28.916+0000 7f3349b28700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:34:34.646 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:34 smithi106 podman[72480]: 2024-03-11 19:34:34.352405259 +0000 UTC m=+5.543794407 container died 8ffb48423c34c657723df4e79d36c711ddbdbe180dd312c06588a02a7babc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2) 2024-03-11T19:34:34.981 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:34 smithi106 podman[72480]: 2024-03-11 19:34:34.77642476 +0000 UTC m=+5.967813900 container cleanup 8ffb48423c34c657723df4e79d36c711ddbdbe180dd312c06588a02a7babc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T19:34:34.982 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:34 smithi106 bash[72480]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1 2024-03-11T19:34:35.391 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:34 smithi106 podman[72498]: 2024-03-11 19:34:34.977752138 +0000 UTC m=+0.625453509 container remove 8ffb48423c34c657723df4e79d36c711ddbdbe180dd312c06588a02a7babc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-11T19:34:35.645 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.291173538 +0000 UTC m=+0.017766079 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:34:35.646 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.412053793 +0000 UTC m=+0.138646314 container create e4aa9aff7b8e3ed0489c4e80424bcf676d869c1c63f246e943c533954a874253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:34:35.977 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.673140167 +0000 UTC m=+0.399732696 container init e4aa9aff7b8e3ed0489c4e80424bcf676d869c1c63f246e943c533954a874253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1-deactivate, ceph=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-11T19:34:35.977 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.676394264 +0000 UTC m=+0.402986793 container start e4aa9aff7b8e3ed0489c4e80424bcf676d869c1c63f246e943c533954a874253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:34:35.977 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.773435746 +0000 UTC m=+0.500028278 container attach e4aa9aff7b8e3ed0489c4e80424bcf676d869c1c63f246e943c533954a874253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:34:36.288 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 11 19:34:35 smithi106 podman[72588]: 2024-03-11 19:34:35.973839643 +0000 UTC m=+0.700432173 container died e4aa9aff7b8e3ed0489c4e80424bcf676d869c1c63f246e943c533954a874253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:34:36.600 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.1.service' 2024-03-11T19:34:36.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:36.641 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-11T19:34:36.641 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-11T19:34:36.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.2 2024-03-11T19:34:36.940 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:36 smithi143 systemd[1]: Stopping Ceph osd.2 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:37.440 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2[41124]: 2024-03-11T19:34:36.961+0000 7fe0d9886700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:37.440 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2[41124]: 2024-03-11T19:34:36.961+0000 7fe0d9886700 -1 osd.2 107 *** Got signal Terminated *** 2024-03-11T19:34:37.440 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2[41124]: 2024-03-11T19:34:36.961+0000 7fe0d9886700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:34:42.598 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:42 smithi143 podman[73436]: 2024-03-11 19:34:42.275501282 +0000 UTC m=+5.444501919 container died 60f7effeec28607cdc45fd990a2163c79d89c819ea2924eb23239c3118044066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-11T19:34:42.940 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:42 smithi143 podman[73436]: 2024-03-11 19:34:42.693978789 +0000 UTC m=+5.862979423 container cleanup 60f7effeec28607cdc45fd990a2163c79d89c819ea2924eb23239c3118044066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:34:42.940 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:42 smithi143 bash[73436]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2 2024-03-11T19:34:43.308 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:42 smithi143 podman[73454]: 2024-03-11 19:34:42.94563539 +0000 UTC m=+0.659822401 container remove 60f7effeec28607cdc45fd990a2163c79d89c819ea2924eb23239c3118044066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-11T19:34:43.603 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.207103417 +0000 UTC m=+0.017739643 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:34:43.604 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.329822083 +0000 UTC m=+0.140458294 container create 3d2b00fe68346feb5f3a552d9a8c208edeceab62c8d48b6aad5b7c73de3f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-11T19:34:43.897 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.599898769 +0000 UTC m=+0.410534995 container init 3d2b00fe68346feb5f3a552d9a8c208edeceab62c8d48b6aad5b7c73de3f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-11T19:34:43.897 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.603530741 +0000 UTC m=+0.414166959 container start 3d2b00fe68346feb5f3a552d9a8c208edeceab62c8d48b6aad5b7c73de3f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2-deactivate, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-11T19:34:43.897 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.708152332 +0000 UTC m=+0.518788559 container attach 3d2b00fe68346feb5f3a552d9a8c208edeceab62c8d48b6aad5b7c73de3f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:34:44.190 INFO:journalctl@ceph.osd.2.smithi143.stdout:Mar 11 19:34:43 smithi143 podman[73545]: 2024-03-11 19:34:43.893589334 +0000 UTC m=+0.704225565 container died 3d2b00fe68346feb5f3a552d9a8c208edeceab62c8d48b6aad5b7c73de3f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-11T19:34:44.563 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.2.service' 2024-03-11T19:34:44.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:44.608 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-11T19:34:44.608 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-11T19:34:44.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.3 2024-03-11T19:34:45.150 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:44 smithi072 systemd[1]: Stopping Ceph osd.3 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:45.430 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:45 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:34:45.148+0000 7f2d51000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:45.431 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:45 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:34:45.148+0000 7f2d51000700 -1 osd.3 107 *** Got signal Terminated *** 2024-03-11T19:34:45.431 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:45 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:34:45.148+0000 7f2d51000700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:34:46.528 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:46 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:46.254+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:47.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:46.859+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:47.680 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:47 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:47.260+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:48.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:47.864+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:48.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:47.871+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:48.681 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:48 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:34:48.534+0000 7f2d47800700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:23.376396+0000 front 2024-03-11T19:34:23.376661+0000 (oldest deadline 2024-03-11T19:34:48.076600+0000) 2024-03-11T19:34:48.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:48 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:48.291+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:48.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:48 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:48.510+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:48.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:48.331+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:49.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:48.857+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:49.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:48.901+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:49.521 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:49.259+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:49.364+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:49.930 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3[52942]: 2024-03-11T19:34:49.569+0000 7f2d47800700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:23.376396+0000 front 2024-03-11T19:34:23.376661+0000 (oldest deadline 2024-03-11T19:34:48.076600+0000) 2024-03-11T19:34:49.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:49 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:49.517+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:50.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:49.848+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:50.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:49.898+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:50.680 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:50.304+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:50.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:50 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:50.504+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:50.689 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:50.368+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:50.977 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:50 smithi072 podman[93011]: 2024-03-11 19:34:50.720982045 +0000 UTC m=+5.706769895 container died 9fe7d2b3957b8c22d360d0d240679e748a65d42d43573e24d275a6ad927004dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-11T19:34:51.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:50.873+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:51.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:50 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:50.939+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:51.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:51 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:51.268+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:51.681 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:51 smithi072 podman[93011]: 2024-03-11 19:34:51.245136235 +0000 UTC m=+6.230924129 container cleanup 9fe7d2b3957b8c22d360d0d240679e748a65d42d43573e24d275a6ad927004dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-11T19:34:51.681 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:51 smithi072 bash[93011]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3 2024-03-11T19:34:51.681 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:51 smithi072 podman[93041]: 2024-03-11 19:34:51.491592479 +0000 UTC m=+0.764069449 container remove 9fe7d2b3957b8c22d360d0d240679e748a65d42d43573e24d275a6ad927004dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:34:51.682 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:51 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:51.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:51.407+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:52.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:51.875+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:52.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:51 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:51.896+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:52.523 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:52 smithi072 podman[93127]: 2024-03-11 19:34:52.227395248 +0000 UTC m=+0.038686840 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:34:52.523 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:52 smithi072 podman[93127]: 2024-03-11 19:34:52.368437259 +0000 UTC m=+0.179728897 container create 96f0653328558dcd178e74aea876d85ab182ab737d047811e25024730723b41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-11T19:34:52.523 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:52.455+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:52.523 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:52 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:52.272+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:52.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:52.425+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:52.931 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:52 smithi072 podman[93127]: 2024-03-11 19:34:52.649866206 +0000 UTC m=+0.461157868 container init 96f0653328558dcd178e74aea876d85ab182ab737d047811e25024730723b41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-11T19:34:52.931 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:52 smithi072 podman[93127]: 2024-03-11 19:34:52.662407384 +0000 UTC m=+0.473698980 container start 96f0653328558dcd178e74aea876d85ab182ab737d047811e25024730723b41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:34:52.931 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:52 smithi072 podman[93127]: 2024-03-11 19:34:52.76627374 +0000 UTC m=+0.577565358 container attach 96f0653328558dcd178e74aea876d85ab182ab737d047811e25024730723b41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-11T19:34:53.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:52 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:52.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:53.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:52.854+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:53.432 INFO:journalctl@ceph.osd.3.smithi072.stdout:Mar 11 19:34:53 smithi072 podman[93127]: 2024-03-11 19:34:53.088569202 +0000 UTC m=+0.899860837 container died 96f0653328558dcd178e74aea876d85ab182ab737d047811e25024730723b41f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:34:53.432 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:53 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:53.270+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:53.850 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:53.451+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:53.853 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:53 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:53.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:53.907 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.3.service' 2024-03-11T19:34:54.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:34:54.018 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-11T19:34:54.018 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-11T19:34:54.019 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.4 2024-03-11T19:34:54.145 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:54 smithi106 systemd[1]: Stopping Ceph osd.4 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:34:54.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:53 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:53.852+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:54.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:53.845+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:54.645 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:34:54.345+0000 7f1b9ac38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:34:54.646 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:34:54.345+0000 7f1b9ac38700 -1 osd.4 107 *** Got signal Terminated *** 2024-03-11T19:34:54.646 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:34:54.345+0000 7f1b9ac38700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:34:54.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:54 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:54.310+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:54.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:54 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:54.413+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:54.830 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:54.493+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:55.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:54 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:54.839+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:55.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:54.825+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:55.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:55 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:55.278+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:55.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:55 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:55.410+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:55.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:55 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:55.410+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:34:55.843 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:55.487+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:56.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:55 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:55.834+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:56.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:55.839+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:56.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:56.234+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:56.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:56.373+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:56.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:56 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:56.373+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:34:56.849 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:56.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:57.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:56.823+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:57.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:56 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:56.823+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:34:57.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:56.871+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:57.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:34:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:34:56.844+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:34:57.680 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:57 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:57.222+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:57.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:57 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:57.222+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:34:57.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:57 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:57.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:57.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:57 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:57.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:34:57.818 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:57.464+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:58.146 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:34:57.799+0000 7f1b93224700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.815856+0000 front 2024-03-11T19:34:32.815774+0000 (oldest deadline 2024-03-11T19:34:57.515947+0000) 2024-03-11T19:34:58.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:57.825+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:58.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:57 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:57.825+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:34:58.154 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:34:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:34:57.813+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:34:58.154 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:57.878+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:58.155 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:57.878+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:34:58.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:58 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:58.236+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:58.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:58 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:58.236+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:34:58.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:58 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:58.381+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:58.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:58 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:58.381+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:34:58.815 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:58.469+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:58.816 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:58.469+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:34:59.145 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4[47021]: 2024-03-11T19:34:58.767+0000 7f1b93224700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.815856+0000 front 2024-03-11T19:34:32.815774+0000 (oldest deadline 2024-03-11T19:34:57.515947+0000) 2024-03-11T19:34:59.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:58.828+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:34:59.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:58 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:58.828+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:34:59.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:34:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:34:58.811+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:34:59.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:58.891+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:34:59.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:58 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:58.891+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:34:59.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:59.226+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:34:59.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:34:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:34:59.226+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:34:59.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:59.385+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:34:59.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:34:59 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:34:59.385+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:34:59.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:59 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:59.421+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:34:59.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:34:59 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:34:59.421+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:00.068 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:34:59 smithi106 podman[72699]: 2024-03-11 19:34:59.76807638 +0000 UTC m=+5.559483951 container died b29cf4e67508f11f644adb2d5a814b3690c95940147833dcb59b30a46545ce8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-11T19:35:00.068 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:59 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:59.821+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:00.069 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:34:59 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:34:59.821+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:00.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:34:59 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:34:59.816+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:00.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:59 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:59.856+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:00.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:34:59 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:34:59.856+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:00.395 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:00.191+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:00.396 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72699]: 2024-03-11 19:35:00.163684516 +0000 UTC m=+5.955092096 container cleanup b29cf4e67508f11f644adb2d5a814b3690c95940147833dcb59b30a46545ce8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-11T19:35:00.396 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 bash[72699]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4 2024-03-11T19:35:00.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:00.205+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:00.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:00.205+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:00.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:00.380+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:00.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:00.380+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:00.727 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72728]: 2024-03-11 19:35:00.431993231 +0000 UTC m=+0.659115698 container remove b29cf4e67508f11f644adb2d5a814b3690c95940147833dcb59b30a46545ce8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-11T19:35:00.793 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:00.461+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:00.793 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:00.461+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:01.055 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:00.836+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:01.055 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:00 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:00.836+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:01.056 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72818]: 2024-03-11 19:35:00.724075159 +0000 UTC m=+0.109192116 container create dd5c18fc5639ff0bab4ce6581251df5afbb033bea7c8735779c135ad5a58653e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397) 2024-03-11T19:35:01.056 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72818]: 2024-03-11 19:35:00.632554889 +0000 UTC m=+0.017671859 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:01.056 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72818]: 2024-03-11 19:35:00.951668458 +0000 UTC m=+0.336785423 container init dd5c18fc5639ff0bab4ce6581251df5afbb033bea7c8735779c135ad5a58653e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-11T19:35:01.056 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:00 smithi106 podman[72818]: 2024-03-11 19:35:00.954840632 +0000 UTC m=+0.339957589 container start dd5c18fc5639ff0bab4ce6581251df5afbb033bea7c8735779c135ad5a58653e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:01.056 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:01 smithi106 podman[72818]: 2024-03-11 19:35:01.05165984 +0000 UTC m=+0.436776801 container attach dd5c18fc5639ff0bab4ce6581251df5afbb033bea7c8735779c135ad5a58653e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:35:01.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:00.789+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:01.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:00.836+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:01.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:00 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:00.836+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:01.396 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 11 19:35:01 smithi106 podman[72818]: 2024-03-11 19:35:01.240033936 +0000 UTC m=+0.625150908 container died dd5c18fc5639ff0bab4ce6581251df5afbb033bea7c8735779c135ad5a58653e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-11T19:35:01.396 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:01.241+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:01.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:01.207+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:01.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:01.207+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:01.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:01.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:01.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:01 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:01.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:01.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:01.504+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:01.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:01.504+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:01.921 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.4.service' 2024-03-11T19:35:01.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:01.968 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-11T19:35:01.968 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-11T19:35:01.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.5 2024-03-11T19:35:02.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:01.817+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:02.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:01 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:01.817+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:02.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:01.802+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:02.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:01.886+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:02.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:01 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:01.886+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:02.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 systemd[1]: Stopping Ceph osd.5 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:02.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:02.253+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:02.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:02.213+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:02.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:02.213+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:02.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:02.405+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:02.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:02 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:02.405+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:02.690 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:02.298+0000 7f46a2549700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:02.690 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:02.298+0000 7f46a2549700 -1 osd.5 107 *** Got signal Terminated *** 2024-03-11T19:35:02.690 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:02.298+0000 7f46a2549700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:02.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:02.512+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:02.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:02.512+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:03.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:02.824+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:03.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:02 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:02.824+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:03.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:02.817+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:03.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:02.918+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:03.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:02 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:02.918+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:03.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:03.244+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:03.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:03.244+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:03.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:03.190+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:03.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:03.190+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:03.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:03.416+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:03.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:03.416+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:03.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:03 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:03.416+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:03.812 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:03.508+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:03.812 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:03.508+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:04.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:03.837+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:04.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:03 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:03.837+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:04.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:03.808+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:04.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:03.907+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:04.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:03.907+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:04.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:03 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:03.907+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:37.695896+0000 front 2024-03-11T19:34:37.695875+0000 (oldest deadline 2024-03-11T19:35:02.996011+0000) 2024-03-11T19:35:04.412 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:04.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:04.412 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:04.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:04.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:04.266+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:04.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:04.266+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:04.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:04.408+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:04.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:04.408+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:04.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:04 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:04.408+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:04.920 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:04.776+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:04.920 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:04.543+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:04.920 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:04.543+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:05.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:04.839+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:05.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:04 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:04.839+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:05.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:04.915+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:05.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:04.915+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:05.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:04 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:04.915+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:37.695896+0000 front 2024-03-11T19:34:37.695875+0000 (oldest deadline 2024-03-11T19:35:02.996011+0000) 2024-03-11T19:35:05.436 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:05.179+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:05.436 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:05.179+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:05.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:05.243+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:05.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:05.243+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:05.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:05.431+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:05.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:05.431+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:05.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:05 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:05.431+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:05.940 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:05.731+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:05.940 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:05.506+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:05.940 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:05.506+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:06.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:05.889+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:06.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:05 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:05.889+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:06.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:06.175+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:06.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:06.175+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:06.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:06.175+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:06.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:06.403+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:06.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:06.403+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:06.432 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:06.403+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:06.440 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:05.946+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:06.440 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:05.946+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:06.440 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:05 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:05.946+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:37.695896+0000 front 2024-03-11T19:34:37.695875+0000 (oldest deadline 2024-03-11T19:35:02.996011+0000) 2024-03-11T19:35:06.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:06.239+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:06.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:06.239+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:06.777 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:06.503+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:06.777 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:06.503+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:07.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:06.869+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:07.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:06.869+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:07.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:06 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:06.869+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:07.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:06.773+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:07.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:06.898+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:20.694360+0000 front 2024-03-11T19:34:20.694629+0000 (oldest deadline 2024-03-11T19:34:46.594224+0000) 2024-03-11T19:35:07.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:06.898+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.595684+0000 front 2024-03-11T19:34:33.595783+0000 (oldest deadline 2024-03-11T19:34:57.695595+0000) 2024-03-11T19:35:07.190 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:06 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5[47676]: 2024-03-11T19:35:06.898+0000 7f469ab35700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:37.695896+0000 front 2024-03-11T19:34:37.695875+0000 (oldest deadline 2024-03-11T19:35:02.996011+0000) 2024-03-11T19:35:07.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:07.261+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:07.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:07.261+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:07.206+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:07.206+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:07.206+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:07.387+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:07.387+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:07.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:07 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:07.387+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:07.806 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:07 smithi143 podman[73666]: 2024-03-11 19:35:07.648752039 +0000 UTC m=+5.486953180 container died c6012eb15228e730e9cd54a2c017fa1378d7077a1da02ae37bd0a8b386ba187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True) 2024-03-11T19:35:07.806 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:07 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:07.524+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:07.806 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:07 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:07.524+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:07.807 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:07 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:07.524+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:08.126 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:07 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:07.802+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:08.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:07.873+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:08.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:07.873+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:08.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:07 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:07.873+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:08.419 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:08 smithi143 podman[73666]: 2024-03-11 19:35:08.122692809 +0000 UTC m=+5.960893949 container cleanup c6012eb15228e730e9cd54a2c017fa1378d7077a1da02ae37bd0a8b386ba187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-11T19:35:08.419 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:08 smithi143 bash[73666]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5 2024-03-11T19:35:08.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:08 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:08.254+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:08.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:08 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:08.254+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:08.200+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:08.200+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:08.200+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:08.382+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:08.382+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:08.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:08 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:08.382+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:08.690 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:08 smithi143 podman[73681]: 2024-03-11 19:35:08.415567252 +0000 UTC m=+0.765566360 container remove c6012eb15228e730e9cd54a2c017fa1378d7077a1da02ae37bd0a8b386ba187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T19:35:08.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:08 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:08.488+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:08.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:08 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:08.488+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:08.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:08 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:08.488+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:09.014 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:08 smithi143 podman[73772]: 2024-03-11 19:35:08.642461135 +0000 UTC m=+0.017669224 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:09.014 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:08 smithi143 podman[73772]: 2024-03-11 19:35:08.741488044 +0000 UTC m=+0.116696138 container create 05a901ee7500c74a6f7aefb02b0476891f4a4e53e8116c77e7612f9fd82dfdef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397) 2024-03-11T19:35:09.015 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:09 smithi143 podman[73772]: 2024-03-11 19:35:09.010437045 +0000 UTC m=+0.385645109 container init 05a901ee7500c74a6f7aefb02b0476891f4a4e53e8116c77e7612f9fd82dfdef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T19:35:09.015 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:08 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:08.842+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:09.015 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:08 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:08.842+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:09.268 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:08 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:08.917+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:09.268 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:08 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:08.917+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:09.268 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:08 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:08.917+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:09.303 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:09 smithi143 podman[73772]: 2024-03-11 19:35:09.013881744 +0000 UTC m=+0.389089829 container start 05a901ee7500c74a6f7aefb02b0476891f4a4e53e8116c77e7612f9fd82dfdef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-11T19:35:09.303 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:09 smithi143 podman[73772]: 2024-03-11 19:35:09.08552322 +0000 UTC m=+0.460731321 container attach 05a901ee7500c74a6f7aefb02b0476891f4a4e53e8116c77e7612f9fd82dfdef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:35:09.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:09 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:09.264+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:09.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:09 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:09.264+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:09.222+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:09.222+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:09.222+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:09.336+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:09.336+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:09.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:09 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:09.336+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:09.684 INFO:journalctl@ceph.osd.5.smithi143.stdout:Mar 11 19:35:09 smithi143 podman[73772]: 2024-03-11 19:35:09.300355458 +0000 UTC m=+0.675563534 container died 05a901ee7500c74a6f7aefb02b0476891f4a4e53e8116c77e7612f9fd82dfdef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-11T19:35:09.684 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:09 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:09.458+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:09.684 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:09 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:09.458+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:09.684 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:09 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:09.458+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:09.940 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:09 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:09.807+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:09.940 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:09 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:09.807+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:10.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.5.service' 2024-03-11T19:35:10.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:10.050 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-11T19:35:10.050 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-11T19:35:10.050 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.6 2024-03-11T19:35:10.241 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:09 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:09.960+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:10.241 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:09 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:09.960+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:10.241 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:09 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:09.960+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:10.430 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:10.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:10.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:10.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 systemd[1]: Stopping Ceph osd.6 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.184+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.184+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:10.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.184+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:10.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:10 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:10.236+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:10.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:10 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:10.236+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:10.793 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:10 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:10.456+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:10.793 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:10 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:10.456+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:10.793 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:10 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:10.456+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:10.931 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.602+0000 7f71f1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:10.931 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.602+0000 7f71f1000700 -1 osd.6 107 *** Got signal Terminated *** 2024-03-11T19:35:10.931 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:10 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:10.602+0000 7f71f1000700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:11.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:10 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:10.788+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:11.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:10 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:10.788+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:11.280 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:10 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:10.941+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:11.280 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:10 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:10.941+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:11.280 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:10 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:10.941+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:11.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:11.276+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:11.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:11.276+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:11.680 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:11.217+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:11.217+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:11.217+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:11.310+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:11.310+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:11.310+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:11.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:11.310+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:11.837 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:11.450+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:11.837 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:11.450+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:11.838 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:11.450+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:12.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:11.833+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:12.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:11.833+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:12.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:11 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:11.833+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:12.283 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:11.974+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:12.283 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:11.974+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:12.283 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:11.974+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:12.284 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:11 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:11.974+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:12.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:12 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:12.279+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:12.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:12 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:12.279+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:12.178+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:12.178+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:12.178+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:12.178+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.379333+0000 front 2024-03-11T19:34:46.379427+0000 (oldest deadline 2024-03-11T19:35:11.679633+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:12.343+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:12.343+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:12.343+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:12.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:12 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:12.343+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:12.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:12.433+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:12.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:12.433+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:12.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:12.433+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:13.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:12.873+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:13.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:12.873+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:13.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:12 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:12.873+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:13.281 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.011+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:13.281 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.011+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:13.281 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.011+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:13.282 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.011+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:13.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:13.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:13.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:13.156+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.379333+0000 front 2024-03-11T19:34:46.379427+0000 (oldest deadline 2024-03-11T19:35:11.679633+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:13.321+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:13.321+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:13.321+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:13.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:13 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:13.321+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:13.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:13.277+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:13.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:13.277+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:13.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:13.277+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:13.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:13.388+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:13.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:13.388+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:13.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:13.388+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:14.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:13.847+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:14.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:13.847+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:14.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:13 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:13.847+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:14.275 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.972+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:14.275 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.972+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:14.275 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.972+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:14.275 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:13 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:13.972+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:14.276 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:14.271+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:14.276 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:14.271+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:14.276 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:14.271+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:14.133+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:14.133+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:14.133+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:14.133+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.379333+0000 front 2024-03-11T19:34:46.379427+0000 (oldest deadline 2024-03-11T19:35:11.679633+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:14.295+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:14.295+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:14.295+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:14.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:14 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:14.295+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:14.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:14.372+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:14.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:14.372+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:14.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:14.372+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:15.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:14.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:15.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:14.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:15.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:14 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:14.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:15.297 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:14.947+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:15.298 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:14.947+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:15.298 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:14.947+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:15.298 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:14 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:14.947+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:15.430 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:15.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:15.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:15.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:15.340+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:15.145+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.669250+0000 front 2024-03-11T19:34:22.669178+0000 (oldest deadline 2024-03-11T19:34:45.569190+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:15.145+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.974144+0000 front 2024-03-11T19:34:32.974219+0000 (oldest deadline 2024-03-11T19:34:57.074418+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:15.145+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.776846+0000 front 2024-03-11T19:34:41.776802+0000 (oldest deadline 2024-03-11T19:35:05.277002+0000) 2024-03-11T19:35:15.431 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6[59941]: 2024-03-11T19:35:15.145+0000 7f71e7800700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.379333+0000 front 2024-03-11T19:34:46.379427+0000 (oldest deadline 2024-03-11T19:35:11.679633+0000) 2024-03-11T19:35:15.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:15.292+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:15.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:15.292+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:15.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:15.292+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:15.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:15.337+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:15.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:15.337+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:15.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:15.337+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:16.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:15.910+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:16.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:15.910+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:16.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:15 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:15.910+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:16.332 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:16 smithi072 podman[93232]: 2024-03-11 19:35:16.022104867 +0000 UTC m=+5.549711840 container died ff8250d8036231e2bf92cf97980d4d6c535b5451efa032fa6d6f50272a85212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-11T19:35:16.336 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:15.912+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:16.336 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:15.912+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:16.336 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:15.912+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:16.337 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:15 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:15.912+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:16.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:16.332+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:16.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:16.332+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:16.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:16.332+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:16.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:16 smithi072 podman[93232]: 2024-03-11 19:35:16.450874404 +0000 UTC m=+5.978481421 container cleanup ff8250d8036231e2bf92cf97980d4d6c535b5451efa032fa6d6f50272a85212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-11T19:35:16.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:16 smithi072 bash[93232]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6 2024-03-11T19:35:16.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:16 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:16.366+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:16.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:16 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:16.366+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:16.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:16 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:16.366+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:16.682 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:16 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:16.366+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:16.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:16.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:16.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:16.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:16.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:16.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:17.180 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:16 smithi072 podman[93261]: 2024-03-11 19:35:16.711565345 +0000 UTC m=+0.674760296 container remove ff8250d8036231e2bf92cf97980d4d6c535b5451efa032fa6d6f50272a85212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-11T19:35:17.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:16.888+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:17.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:16.888+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:17.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:16 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:16.888+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:17.300 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:16.948+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:17.300 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:16.948+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:17.300 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:16.948+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:17.300 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:16 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:16.948+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:17.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:17.296+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:17.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:17.296+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:17.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:17.296+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:17 smithi072 podman[93352]: 2024-03-11 19:35:17.352668153 +0000 UTC m=+0.043151109 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:17 smithi072 podman[93352]: 2024-03-11 19:35:17.563585781 +0000 UTC m=+0.254068737 container create 00c901cca81c3cb9a03525fbf1f42002fdf16ac132350762f28666b84bf7a987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:17 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:17.391+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:17 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:17.391+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:17 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:17.391+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:17.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:17 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:17.391+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:17.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:17.357+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:17.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:17.357+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:17.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:17.357+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:18.181 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:17 smithi072 podman[93352]: 2024-03-11 19:35:17.836925686 +0000 UTC m=+0.527408695 container init 00c901cca81c3cb9a03525fbf1f42002fdf16ac132350762f28666b84bf7a987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True) 2024-03-11T19:35:18.181 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:17 smithi072 podman[93352]: 2024-03-11 19:35:17.847727509 +0000 UTC m=+0.538210501 container start 00c901cca81c3cb9a03525fbf1f42002fdf16ac132350762f28666b84bf7a987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397) 2024-03-11T19:35:18.181 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:17 smithi072 podman[93352]: 2024-03-11 19:35:17.93628986 +0000 UTC m=+0.626772860 container attach 00c901cca81c3cb9a03525fbf1f42002fdf16ac132350762f28666b84bf7a987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397) 2024-03-11T19:35:18.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:17.867+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:18.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:17.867+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:18.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:17 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:17.867+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:18.264 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:17.959+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:18.264 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:17.959+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:18.264 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:17.959+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:18.264 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:17 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:17.959+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:18.463 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:18.355+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:18.463 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:18.355+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:18.463 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:18.355+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:18.607 INFO:journalctl@ceph.osd.6.smithi072.stdout:Mar 11 19:35:18 smithi072 podman[93378]: 2024-03-11 19:35:18.324279593 +0000 UTC m=+0.035526419 container died 00c901cca81c3cb9a03525fbf1f42002fdf16ac132350762f28666b84bf7a987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-11T19:35:18.608 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:18 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:18.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:18.608 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:18 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:18.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:18.608 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:18 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:18.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:18.608 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:18 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:18.395+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:18.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:18.260+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:18.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:18.260+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:18.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:18.260+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:19.016 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.6.service' 2024-03-11T19:35:19.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:19.140 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-11T19:35:19.140 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-11T19:35:19.140 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.7 2024-03-11T19:35:19.168 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:18.914+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:19.168 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:18.914+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:19.168 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:18.914+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:19.169 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:18 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:18.914+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:19.189 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:18.890+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:19.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:18.890+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:19.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:18 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:18.890+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:19.437 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 systemd[1]: Stopping Ceph osd.7 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:19.437 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.236+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:19.437 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.236+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:19.437 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.236+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:19.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:19.379+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:19.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:19.379+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:19.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:19.379+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:19.895 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.432+0000 7f685e1c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:19.896 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.432+0000 7f685e1c4700 -1 osd.7 107 *** Got signal Terminated *** 2024-03-11T19:35:19.896 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:19.432+0000 7f685e1c4700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:19.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:19 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:19.427+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:19.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:19 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:19.427+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:19.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:19 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:19.427+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:19.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:19 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:19.427+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:20.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:19.870+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:20.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:19.870+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:20.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:19 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:19.870+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:20.226 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:19.937+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:20.226 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:19.937+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:20.226 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:19.937+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:20.226 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:19.937+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:20.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:20.222+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:20.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:20.222+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:20.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:20.222+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:20.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:20.342+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:20.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:20.342+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:20.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:20.342+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:20.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:20.342+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:20.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:20.428+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:20.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:20.428+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:20.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:20.428+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:20.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:20 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:20.428+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:21.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:20.915+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:21.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:20.915+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:21.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:20 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:20.915+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:21.235 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:20.928+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:21.235 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:20.928+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:21.235 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:20.928+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:21.235 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:20 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:20.928+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:21.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:21.231+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:21.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:21.231+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:21.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:21.231+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:21.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:21.336+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:21.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:21.336+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:21.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:21.336+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:21.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:21.336+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:21.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:21.439+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:21.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:21.439+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:21.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:21.439+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:21.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:21 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:21.439+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:22.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:21.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:22.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:21.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:22.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:21.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:22.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:21.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:22.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:21 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:21.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:22.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:21.892+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:22.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:21.892+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:22.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:21 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:21.892+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:22.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:22.226+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:22.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:22.226+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:22.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:22.226+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:22.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:22.299+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:22.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:22.299+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:22.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:22.299+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:22.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:22.299+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:22.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:22.473+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:22.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:22.473+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:22.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:22.473+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:22.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:22 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:22.473+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:23.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:22.880+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:23.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:22.880+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:23.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:22.880+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:23.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:22.880+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:23.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:22.880+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:23.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:22.925+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:23.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:22.925+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:23.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:22 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:22.925+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:23.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:23.182+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:23.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:23.182+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:23.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:23.182+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:23.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:23.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:23.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:23.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:23.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:23.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:23.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:23.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:23.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:23.464+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:23.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:23.464+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:23.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:23.464+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:23.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:23.464+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:23.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:23 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:23.464+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:24.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:23.884+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:24.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:23.884+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:24.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:23.884+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:24.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:23.884+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:24.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:23 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:23.884+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:24.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:23.889+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:24.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:23.889+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:24.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:23.889+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:24.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:23 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:23.889+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:24.645 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:24.158+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.863961+0000 front 2024-03-11T19:34:33.864005+0000 (oldest deadline 2024-03-11T19:34:59.764163+0000) 2024-03-11T19:35:24.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:24.158+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:39.764667+0000 front 2024-03-11T19:34:39.764721+0000 (oldest deadline 2024-03-11T19:35:02.664703+0000) 2024-03-11T19:35:24.646 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7[53705]: 2024-03-11T19:35:24.158+0000 7f68567b0700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.866251+0000 front 2024-03-11T19:34:47.866151+0000 (oldest deadline 2024-03-11T19:35:13.166101+0000) 2024-03-11T19:35:24.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:24.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:24.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:24.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:24.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:24.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:24.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:24.306+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:24.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:24.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:24.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:24.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:24.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:24.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:24.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:24.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:24.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:24 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:24.435+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:25.037 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:24 smithi106 podman[72923]: 2024-03-11 19:35:24.763880428 +0000 UTC m=+5.437305375 container died 4e60e5997e015d4fe5f3d0465339aec4fa0a9393d4fb3c8fd5dda1489b0ba7dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:35:25.037 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:24.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:25.037 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:24.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:25.038 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:24.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:25.038 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:24.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:25.038 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:24 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:24.847+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:25.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:24.897+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:25.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:24.897+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:25.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:24.897+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:25.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:24 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:24.897+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:25.395 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:25 smithi106 podman[72923]: 2024-03-11 19:35:25.133085637 +0000 UTC m=+5.806510575 container cleanup 4e60e5997e015d4fe5f3d0465339aec4fa0a9393d4fb3c8fd5dda1489b0ba7dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T19:35:25.395 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:25 smithi106 bash[72923]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7 2024-03-11T19:35:25.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:25.406+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:25.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:25.406+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:25.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:25.406+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:25.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:25.406+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:25.682 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:25 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:25.406+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:25.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:25.326+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:25.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:25.326+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:25.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:25.326+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:25.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:25.326+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:25.798 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:25 smithi106 podman[72940]: 2024-03-11 19:35:25.459904384 +0000 UTC m=+0.693583026 container remove 4e60e5997e015d4fe5f3d0465339aec4fa0a9393d4fb3c8fd5dda1489b0ba7dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:35:26.067 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:25 smithi106 podman[73033]: 2024-03-11 19:35:25.697812207 +0000 UTC m=+0.017745181 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:26.067 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:25 smithi106 podman[73033]: 2024-03-11 19:35:25.810615367 +0000 UTC m=+0.130548322 container create 8dd17165858e5e96ecc615b22886986d70608d9138df0899833520f8199bb986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-11T19:35:26.067 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:26 smithi106 podman[73033]: 2024-03-11 19:35:26.063664743 +0000 UTC m=+0.383597704 container init 8dd17165858e5e96ecc615b22886986d70608d9138df0899833520f8199bb986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:26.067 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:25 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:25.872+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:26.067 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:25 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:25.872+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:26.068 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:25 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:25.872+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:26.068 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:25 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:25.872+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:26.068 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:25 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:25.872+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:26.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:25.856+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:26.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:25.856+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:26.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:25.856+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:26.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:25 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:25.856+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:26.361 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:26 smithi106 podman[73033]: 2024-03-11 19:35:26.066915657 +0000 UTC m=+0.386848622 container start 8dd17165858e5e96ecc615b22886986d70608d9138df0899833520f8199bb986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:35:26.361 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:26 smithi106 podman[73033]: 2024-03-11 19:35:26.146763778 +0000 UTC m=+0.466696750 container attach 8dd17165858e5e96ecc615b22886986d70608d9138df0899833520f8199bb986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7-deactivate, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-11T19:35:26.625 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 11 19:35:26 smithi106 podman[73033]: 2024-03-11 19:35:26.3573334 +0000 UTC m=+0.677266372 container died 8dd17165858e5e96ecc615b22886986d70608d9138df0899833520f8199bb986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-11T19:35:26.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:26.450+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:26.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:26.450+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:26.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:26.450+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:26.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:26.450+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:26.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:26 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:26.450+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:26.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:26.374+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:26.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:26.374+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:26.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:26.374+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:26.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:26.374+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:27.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.7.service' 2024-03-11T19:35:27.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:27.102 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-11T19:35:27.102 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-11T19:35:27.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.8 2024-03-11T19:35:27.132 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:26.845+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:27.133 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:26.845+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:27.133 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:26.845+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:27.133 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:26 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:26.845+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:27.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:26.878+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:27.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:26.878+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:27.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:26.878+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:27.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:26.878+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:27.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:26.878+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:27.392 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 systemd[1]: Stopping Ceph osd.8 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:27.690 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.407+0000 7eff2d371700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:27.691 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.407+0000 7eff2d371700 -1 osd.8 107 *** Got signal Terminated *** 2024-03-11T19:35:27.691 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.407+0000 7eff2d371700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:27.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:27.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:27.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:27.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:27.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:27.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:27.692 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:27.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:27.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:27.440+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:27.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:27.440+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:27.932 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:27.440+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:27.932 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:27.440+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:27.932 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:27 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:27.440+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:28.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.885+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:28.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.885+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:28.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.885+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:28.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:27 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:27.885+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:28.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:27 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:27.920+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:28.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:27 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:27.920+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:28.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:27 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:27.920+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:28.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:27 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:27.920+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:28.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:27 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:27.920+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:28.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:28.340+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:28.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:28.340+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:28.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:28.340+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:28.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:28.340+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:28.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:28.340+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:28.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:28.458+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:28.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:28.458+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:28.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:28.458+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:28.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:28.458+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:28.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:28 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:28.458+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:29.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:28.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:29.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:28.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:29.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:28.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:29.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:28 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:28.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:29.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:28 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:28.949+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:29.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:29.370+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:29.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:29.370+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:29.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:29.370+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:29.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:29.370+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:29.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:29.370+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:29.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:29 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:29.470+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:30.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:29.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:30.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:29.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:30.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:29.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:30.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:29 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:29.848+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:30.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:29 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:29.951+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:30.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:30.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:30.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:30.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:30.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:30.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:30.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:30.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:30.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:30.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:30.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:30 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:30.437+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:31.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:30.866+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:31.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:30.866+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:31.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:30.866+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:31.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:30.866+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:31.191 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:30 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:30.866+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:06.024734+0000 front 2024-03-11T19:35:06.024774+0000 (oldest deadline 2024-03-11T19:35:30.124893+0000) 2024-03-11T19:35:31.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:31.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:31.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:31.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:31.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:31.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:30.903+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:31.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:31 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:31.415+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:31.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:31.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:31.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:31.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:31.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:31.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:31.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:31.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:31.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:31.382+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:32.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:31 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:31.875+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:32.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:31.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:32.720207+0000 front 2024-03-11T19:34:32.720177+0000 (oldest deadline 2024-03-11T19:34:56.820144+0000) 2024-03-11T19:35:32.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:31.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.521052+0000 front 2024-03-11T19:34:41.521201+0000 (oldest deadline 2024-03-11T19:35:07.921791+0000) 2024-03-11T19:35:32.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:31.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:47.922559+0000 front 2024-03-11T19:34:47.922424+0000 (oldest deadline 2024-03-11T19:35:11.422285+0000) 2024-03-11T19:35:32.190 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:31.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.823768+0000 front 2024-03-11T19:34:57.823825+0000 (oldest deadline 2024-03-11T19:35:23.123779+0000) 2024-03-11T19:35:32.191 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:31 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8[54245]: 2024-03-11T19:35:31.879+0000 7eff29965700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:06.024734+0000 front 2024-03-11T19:35:06.024774+0000 (oldest deadline 2024-03-11T19:35:30.124893+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:32.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:32 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:32.397+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:32.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:32 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:32.398+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:32.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:32 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:32.398+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:32.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:32 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:32.398+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:32.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:32 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:32.398+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:32.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:32 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:32.398+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:33.238 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:32 smithi143 podman[73889]: 2024-03-11 19:35:32.939103732 +0000 UTC m=+5.647457129 container died e383f6f3bb518562da9e63226140cfa9b23f8c01762a5f48cca960590be69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:35:33.397 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:33.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:33.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:33.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:33.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:33.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:32 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:32.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:33.565 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:33 smithi143 podman[73889]: 2024-03-11 19:35:33.335390924 +0000 UTC m=+6.043744312 container cleanup e383f6f3bb518562da9e63226140cfa9b23f8c01762a5f48cca960590be69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-11T19:35:33.565 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:33 smithi143 bash[73889]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8 2024-03-11T19:35:33.566 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:33 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:33.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:33.566 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:33 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:33.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:33.566 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:33 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:33.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:33.566 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:33 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:33.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:33.566 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:33 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:33.448+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:33.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:33 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:33.446+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:33.940 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:33 smithi143 podman[73907]: 2024-03-11 19:35:33.56179539 +0000 UTC m=+0.622915358 container remove e383f6f3bb518562da9e63226140cfa9b23f8c01762a5f48cca960590be69348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-03-11T19:35:34.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:34.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:34.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:34.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:34.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:34.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:33 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:33.879+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:34.261 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:33 smithi143 podman[73997]: 2024-03-11 19:35:33.89967617 +0000 UTC m=+0.017508004 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:34.261 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:34 smithi143 podman[73997]: 2024-03-11 19:35:34.020941024 +0000 UTC m=+0.138772844 container create c016d5d1b995b8452398cb36c5bffc6f0a394bd8c810e6a0d9fc16fdf423bc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-11T19:35:34.553 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:34 smithi143 podman[73997]: 2024-03-11 19:35:34.25718971 +0000 UTC m=+0.375021543 container init c016d5d1b995b8452398cb36c5bffc6f0a394bd8c810e6a0d9fc16fdf423bc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8-deactivate, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-11T19:35:34.553 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:34 smithi143 podman[73997]: 2024-03-11 19:35:34.26077148 +0000 UTC m=+0.378603296 container start c016d5d1b995b8452398cb36c5bffc6f0a394bd8c810e6a0d9fc16fdf423bc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:35:34.553 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:34 smithi143 podman[73997]: 2024-03-11 19:35:34.373594379 +0000 UTC m=+0.491426217 container attach c016d5d1b995b8452398cb36c5bffc6f0a394bd8c810e6a0d9fc16fdf423bc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-11T19:35:34.554 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:34 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:34.419+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:34.554 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:34 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:34.419+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:34.554 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:34 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:34.419+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:34.554 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:34 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:34.419+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:34.554 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:34 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:34.419+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:34.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:34 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:34.424+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:34.922 INFO:journalctl@ceph.osd.8.smithi143.stdout:Mar 11 19:35:34 smithi143 podman[73997]: 2024-03-11 19:35:34.550194417 +0000 UTC m=+0.668026250 container died c016d5d1b995b8452398cb36c5bffc6f0a394bd8c810e6a0d9fc16fdf423bc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T19:35:35.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.8.service' 2024-03-11T19:35:35.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:35.344 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-11T19:35:35.344 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-11T19:35:35.344 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.9 2024-03-11T19:35:35.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:35.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:35.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:35.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:35.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:35.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:34 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:34.925+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:35.518 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:35.415+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:35.518 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:35.415+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:35.518 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:35.415+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:35.518 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:35.415+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:35.518 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:35.415+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:35.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.410+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:35.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.412+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:35.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.412+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:35.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.412+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:35.681 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.412+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:35.682 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.412+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:35.682 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 systemd[1]: Stopping Ceph osd.9 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:36.181 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.914+0000 7f0486200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:36.181 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.914+0000 7f0486200700 -1 osd.9 107 *** Got signal Terminated *** 2024-03-11T19:35:36.181 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:35 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:35.914+0000 7f0486200700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:36.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:36.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:36.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:36.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:36.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:36.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:35 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:35.953+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:36.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:36.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:36.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:36.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:36.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:36.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:36.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:36.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:36.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:36 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:36.387+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.444+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.444+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.444+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.445+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.445+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:36.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:36 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:36.445+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:37.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:36 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:36.955+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:37.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:37 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:37.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:37.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:37 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:37.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:37.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:37 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:37.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:37.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:37 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:37.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:37.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:37 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:37.356+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:37.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:37 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:37.463+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.005+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:38.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:38.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:38.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:38.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:38.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:38.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:38.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:38.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:38.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:38.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:38.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:38 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:38.468+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:39.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:39.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:39.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:39.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:39.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:39.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:38 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:38.990+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:39.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:39 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:39.346+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:39.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:39 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:39.467+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:40.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:40.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:40.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:40.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:40.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:40.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:39 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:39.993+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:40.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:40 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:40.366+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.024677+0000 front 2024-03-11T19:34:22.024902+0000 (oldest deadline 2024-03-11T19:34:47.924910+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:31.426611+0000 front 2024-03-11T19:34:31.426682+0000 (oldest deadline 2024-03-11T19:34:54.926848+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:40.828616+0000 front 2024-03-11T19:34:40.828556+0000 (oldest deadline 2024-03-11T19:35:02.528791+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:48.932944+0000 front 2024-03-11T19:34:48.932714+0000 (oldest deadline 2024-03-11T19:35:11.232588+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:57.035120+0000 front 2024-03-11T19:34:57.035338+0000 (oldest deadline 2024-03-11T19:35:22.935315+0000) 2024-03-11T19:35:40.931 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:40 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9[66530]: 2024-03-11T19:35:40.505+0000 7f0481a00700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.237908+0000 front 2024-03-11T19:35:05.237976+0000 (oldest deadline 2024-03-11T19:35:28.738141+0000) 2024-03-11T19:35:41.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:41.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:40 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:40.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:41.647 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:41 smithi072 podman[93458]: 2024-03-11 19:35:41.366925342 +0000 UTC m=+5.593366005 container died 6a1ad1cf7a704f5d378e4d54aaab2264789bd9c67418bc4cf39d15152328306c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:41.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:41 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:41.381+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:41.930 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:41 smithi072 podman[93458]: 2024-03-11 19:35:41.794448388 +0000 UTC m=+6.020889085 container cleanup 6a1ad1cf7a704f5d378e4d54aaab2264789bd9c67418bc4cf39d15152328306c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:35:41.930 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:41 smithi072 bash[93458]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:42.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.015+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:42.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:42 smithi072 podman[93477]: 2024-03-11 19:35:42.005207801 +0000 UTC m=+0.622031204 container remove 6a1ad1cf7a704f5d378e4d54aaab2264789bd9c67418bc4cf39d15152328306c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:42.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:42.365+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:43.143 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:42 smithi072 podman[93563]: 2024-03-11 19:35:42.701988013 +0000 UTC m=+0.031850842 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:43.143 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:42 smithi072 podman[93563]: 2024-03-11 19:35:42.956944732 +0000 UTC m=+0.286807598 container create 0167246caf5d5231f0d08bd13457b2f1b7fe29a2985410b8b15ef38246a32250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:43.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:42 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:42.978+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:43.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:43 smithi072 podman[93563]: 2024-03-11 19:35:43.263989863 +0000 UTC m=+0.593852735 container init 0167246caf5d5231f0d08bd13457b2f1b7fe29a2985410b8b15ef38246a32250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-11T19:35:43.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:43 smithi072 podman[93563]: 2024-03-11 19:35:43.275544535 +0000 UTC m=+0.605407407 container start 0167246caf5d5231f0d08bd13457b2f1b7fe29a2985410b8b15ef38246a32250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-11T19:35:43.431 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:43 smithi072 podman[93563]: 2024-03-11 19:35:43.387844817 +0000 UTC m=+0.717707730 container attach 0167246caf5d5231f0d08bd13457b2f1b7fe29a2985410b8b15ef38246a32250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9-deactivate, RELEASE=squid-94a5397, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:43.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:43 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:43.393+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:43.805 INFO:journalctl@ceph.osd.9.smithi072.stdout:Mar 11 19:35:43 smithi072 podman[93563]: 2024-03-11 19:35:43.712738882 +0000 UTC m=+1.042601718 container died 0167246caf5d5231f0d08bd13457b2f1b7fe29a2985410b8b15ef38246a32250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-11T19:35:44.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:44.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:43 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:43.965+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:44.577 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.9.service' 2024-03-11T19:35:44.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:44.685 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-11T19:35:44.685 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-11T19:35:44.686 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.10 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:44.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:44 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:44.351+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:44.982 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:44 smithi106 systemd[1]: Stopping Ceph osd.10 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:44 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.977+0000 7f820fbf3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:44 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.977+0000 7f820fbf3700 -1 osd.10 107 *** Got signal Terminated *** 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:44 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.977+0000 7f820fbf3700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:45.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:44.998+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:45.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:45 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:45.315+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:46.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:46.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:45 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:45.996+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:46.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:46.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:46.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:46.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:46.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:46.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:46.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:46.318+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:47.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:46 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:46.979+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:47.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:47 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:47.303+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:48.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:47 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:47.985+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:48.591 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:48.592 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:48 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:48.263+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:49.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:48 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:48.950+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:49.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:49 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:49.298+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:50.398 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:22.107197+0000 front 2024-03-11T19:34:22.107053+0000 (oldest deadline 2024-03-11T19:34:47.407027+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.308031+0000 front 2024-03-11T19:34:33.308059+0000 (oldest deadline 2024-03-11T19:34:56.808176+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.409728+0000 front 2024-03-11T19:34:41.409553+0000 (oldest deadline 2024-03-11T19:35:06.709581+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-03-11T19:34:46.710122+0000 front 2024-03-11T19:34:46.710217+0000 (oldest deadline 2024-03-11T19:35:11.910475+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:58.211874+0000 front 2024-03-11T19:34:58.211912+0000 (oldest deadline 2024-03-11T19:35:21.111919+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.713356+0000 front 2024-03-11T19:35:05.713374+0000 (oldest deadline 2024-03-11T19:35:28.513485+0000) 2024-03-11T19:35:50.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:49 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10[60586]: 2024-03-11T19:35:49.907+0000 7f820c1e7700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.315448+0000 front 2024-03-11T19:35:14.315364+0000 (oldest deadline 2024-03-11T19:35:40.215054+0000) 2024-03-11T19:35:50.682 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:50 smithi106 podman[73149]: 2024-03-11 19:35:50.416887659 +0000 UTC m=+5.545802733 container died 19492282dd0cce3e0367f3bc46cc523393008ec01e4a018ffd590bb243d0fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:50.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:50.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:50 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:50.313+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:51.057 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:50 smithi106 podman[73149]: 2024-03-11 19:35:50.777728398 +0000 UTC m=+5.906643462 container cleanup 19492282dd0cce3e0367f3bc46cc523393008ec01e4a018ffd590bb243d0fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:51.057 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:50 smithi106 bash[73149]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10 2024-03-11T19:35:51.389 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73167]: 2024-03-11 19:35:51.053764563 +0000 UTC m=+0.635012172 container remove 19492282dd0cce3e0367f3bc46cc523393008ec01e4a018ffd590bb243d0fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-11T19:35:51.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.288549777 +0000 UTC m=+0.017206710 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:51.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.387839078 +0000 UTC m=+0.116496038 container create ae3313ad4ac15bd33b0d3698ccc49ce84fb9bd9a70af22b9283209c7d7f60311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-11T19:35:51.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.598983285 +0000 UTC m=+0.327640203 container init ae3313ad4ac15bd33b0d3698ccc49ce84fb9bd9a70af22b9283209c7d7f60311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-11T19:35:51.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.601940289 +0000 UTC m=+0.330597205 container start ae3313ad4ac15bd33b0d3698ccc49ce84fb9bd9a70af22b9283209c7d7f60311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:51.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:51.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:51 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:51.269+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:52.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.682093957 +0000 UTC m=+0.410750874 container attach ae3313ad4ac15bd33b0d3698ccc49ce84fb9bd9a70af22b9283209c7d7f60311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-11T19:35:52.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 11 19:35:51 smithi106 podman[73260]: 2024-03-11 19:35:51.894803019 +0000 UTC m=+0.623459939 container died ae3313ad4ac15bd33b0d3698ccc49ce84fb9bd9a70af22b9283209c7d7f60311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-11T19:35:52.513 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.10.service' 2024-03-11T19:35:52.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:35:52.555 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-11T19:35:52.556 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-11T19:35:52.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.11 2024-03-11T19:35:52.587 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:52.588 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.312+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:52.868 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 systemd[1]: Stopping Ceph osd.11 for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:35:53.190 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.863+0000 7fd3257b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:35:53.190 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.863+0000 7fd3257b9700 -1 osd.11 107 *** Got signal Terminated *** 2024-03-11T19:35:53.190 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:52 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:52.863+0000 7fd3257b9700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:35:53.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:53.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:53.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:53.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:53.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:53.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:53.691 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:53 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:53.288+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:54.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:54 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:54.251+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:55.690 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:55 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:55.206+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:56.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:56.441 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:56 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:56.164+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:57.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-03-11T19:34:24.217306+0000 front 2024-03-11T19:34:24.217288+0000 (oldest deadline 2024-03-11T19:34:48.317172+0000) 2024-03-11T19:35:57.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-11T19:34:33.518978+0000 front 2024-03-11T19:34:33.518933+0000 (oldest deadline 2024-03-11T19:34:57.618718+0000) 2024-03-11T19:35:57.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-03-11T19:34:41.620124+0000 front 2024-03-11T19:34:41.620200+0000 (oldest deadline 2024-03-11T19:35:07.520224+0000) 2024-03-11T19:35:57.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-11T19:34:54.521867+0000 front 2024-03-11T19:34:54.521812+0000 (oldest deadline 2024-03-11T19:35:19.821735+0000) 2024-03-11T19:35:57.441 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-03-11T19:35:05.023060+0000 front 2024-03-11T19:35:05.023095+0000 (oldest deadline 2024-03-11T19:35:27.923215+0000) 2024-03-11T19:35:57.441 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-03-11T19:35:14.924744+0000 front 2024-03-11T19:35:14.924647+0000 (oldest deadline 2024-03-11T19:35:39.024460+0000) 2024-03-11T19:35:57.441 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-11T19:35:21.925544+0000 front 2024-03-11T19:35:21.925534+0000 (oldest deadline 2024-03-11T19:35:45.425334+0000) 2024-03-11T19:35:57.441 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:57 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11[61427]: 2024-03-11T19:35:57.138+0000 7fd31dda5700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-03-11T19:35:31.326341+0000 front 2024-03-11T19:35:31.326357+0000 (oldest deadline 2024-03-11T19:35:57.126920+0000) 2024-03-11T19:35:58.543 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:58 smithi143 podman[74127]: 2024-03-11 19:35:58.25381115 +0000 UTC m=+5.506112849 container died e63421d357bb02defaa1296b31fbf0893b6374392e418393880f63801ba4a7c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:58.868 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:58 smithi143 podman[74127]: 2024-03-11 19:35:58.621715233 +0000 UTC m=+5.874016935 container cleanup e63421d357bb02defaa1296b31fbf0893b6374392e418393880f63801ba4a7c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:35:58.868 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:58 smithi143 bash[74127]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11 2024-03-11T19:35:59.169 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:58 smithi143 podman[74144]: 2024-03-11 19:35:58.864757351 +0000 UTC m=+0.611828522 container remove e63421d357bb02defaa1296b31fbf0893b6374392e418393880f63801ba4a7c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:59.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.068931306 +0000 UTC m=+0.017932589 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:35:59.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.173832633 +0000 UTC m=+0.122833908 container create f28ad235f6cb9296ac25d0060d272436feb7abefbe96f66fd929946aec0dbd3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T19:35:59.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.376370578 +0000 UTC m=+0.325371851 container init f28ad235f6cb9296ac25d0060d272436feb7abefbe96f66fd929946aec0dbd3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T19:35:59.440 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.379607131 +0000 UTC m=+0.328608409 container start f28ad235f6cb9296ac25d0060d272436feb7abefbe96f66fd929946aec0dbd3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:35:59.844 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.45963374 +0000 UTC m=+0.408635017 container attach f28ad235f6cb9296ac25d0060d272436feb7abefbe96f66fd929946aec0dbd3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-11T19:35:59.844 INFO:journalctl@ceph.osd.11.smithi143.stdout:Mar 11 19:35:59 smithi143 podman[74231]: 2024-03-11 19:35:59.669274093 +0000 UTC m=+0.618275367 container died f28ad235f6cb9296ac25d0060d272436feb7abefbe96f66fd929946aec0dbd3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:00.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@osd.11.service' 2024-03-11T19:36:00.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:00.401 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-11T19:36:00.401 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-11T19:36:00.401 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.a 2024-03-11T19:36:00.931 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:00 smithi072 systemd[1]: Stopping Ceph mds.a for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:01.434 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a[73520]: 2024-03-11T19:36:00.996+0000 7f08c1e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:01.434 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:00 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a[73520]: 2024-03-11T19:36:00.996+0000 7f08c1e00700 -1 mds.a *** got signal Terminated *** 2024-03-11T19:36:05.181 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:05 smithi072 podman[93683]: 2024-03-11 19:36:05.016847915 +0000 UTC m=+4.181072349 container died 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-11T19:36:05.681 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:05 smithi072 podman[93683]: 2024-03-11 19:36:05.425686085 +0000 UTC m=+4.589910570 container cleanup 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:36:05.681 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:05 smithi072 bash[93683]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a 2024-03-11T19:36:06.180 INFO:journalctl@ceph.mds.a.smithi072.stdout:Mar 11 19:36:05 smithi072 podman[93697]: 2024-03-11 19:36:05.69590393 +0000 UTC m=+0.667135424 container remove 9f551f20746b981580e4bf2551f42ed4bcc4fb3a8cfee2e5acadb9d7c8b67b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:06.243 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.a.service' 2024-03-11T19:36:06.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:06.358 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-11T19:36:06.361 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-11T19:36:06.362 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.d 2024-03-11T19:36:06.931 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:06 smithi072 systemd[1]: Stopping Ceph mds.d for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:07.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d[72517]: 2024-03-11T19:36:06.946+0000 7f0f9ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:07.431 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:06 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d[72517]: 2024-03-11T19:36:06.946+0000 7f0f9ea00700 -1 mds.d *** got signal Terminated *** 2024-03-11T19:36:10.492 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:10 smithi072 podman[93805]: 2024-03-11 19:36:10.205568883 +0000 UTC m=+3.375624317 container died dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397) 2024-03-11T19:36:10.808 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:10 smithi072 podman[93805]: 2024-03-11 19:36:10.600625175 +0000 UTC m=+3.770680653 container cleanup dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:10.809 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:10 smithi072 bash[93805]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d 2024-03-11T19:36:11.180 INFO:journalctl@ceph.mds.d.smithi072.stdout:Mar 11 19:36:10 smithi072 podman[93819]: 2024-03-11 19:36:10.804229899 +0000 UTC m=+0.577739252 container remove dfa2c00646d3005a138c723fa92862b0e995aa0a101b123c36acf95630521d2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True) 2024-03-11T19:36:11.263 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.d.service' 2024-03-11T19:36:11.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:11.374 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-11T19:36:11.374 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-11T19:36:11.374 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.g 2024-03-11T19:36:11.681 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:11 smithi072 systemd[1]: Stopping Ceph mds.g for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:12.181 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: 2024-03-11T19:36:11.895+0000 7f7752200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:12.181 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:11 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g[73131]: 2024-03-11T19:36:11.895+0000 7f7752200700 -1 mds.g *** got signal Terminated *** 2024-03-11T19:36:13.930 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:13 smithi072 podman[93926]: 2024-03-11 19:36:13.650223474 +0000 UTC m=+1.872228824 container died a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-11T19:36:14.331 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:14 smithi072 podman[93926]: 2024-03-11 19:36:14.075681112 +0000 UTC m=+2.297686458 container cleanup a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-11T19:36:14.332 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:14 smithi072 bash[93926]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g 2024-03-11T19:36:14.680 INFO:journalctl@ceph.mds.g.smithi072.stdout:Mar 11 19:36:14 smithi072 podman[93941]: 2024-03-11 19:36:14.327774957 +0000 UTC m=+0.661215611 container remove a38a0270e103fc419b2ba21a1ed7f77c2959fbbafdc8aadefad483ee9cabe113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-11T19:36:14.855 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.g.service' 2024-03-11T19:36:14.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:14.969 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-11T19:36:14.969 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-11T19:36:14.970 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.j 2024-03-11T19:36:15.385 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:15 smithi072 systemd[1]: Stopping Ceph mds.j for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:15.680 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j[72152]: 2024-03-11T19:36:15.545+0000 7fb06e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:15.681 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:15 smithi072 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j[72152]: 2024-03-11T19:36:15.545+0000 7fb06e200700 -1 mds.j *** got signal Terminated *** 2024-03-11T19:36:18.254 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:17 smithi072 podman[94048]: 2024-03-11 19:36:17.981150275 +0000 UTC m=+2.551976080 container died 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:36:18.549 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:18 smithi072 podman[94048]: 2024-03-11 19:36:18.359942138 +0000 UTC m=+2.930767936 container cleanup 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-11T19:36:18.549 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:18 smithi072 bash[94048]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j 2024-03-11T19:36:18.930 INFO:journalctl@ceph.mds.j.smithi072.stdout:Mar 11 19:36:18 smithi072 podman[94072]: 2024-03-11 19:36:18.544664908 +0000 UTC m=+0.546198309 container remove 7973b143821e51fe3c2f012423f33976fd3d82adf0a331ce7ce1337d62f63a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-11T19:36:19.023 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.j.service' 2024-03-11T19:36:19.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:19.140 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-11T19:36:19.140 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-11T19:36:19.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.b 2024-03-11T19:36:19.422 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:19 smithi106 systemd[1]: Stopping Ceph mds.b for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:19.688 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: 2024-03-11T19:36:19.418+0000 7f839164b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:19.688 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:19 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b[66291]: 2024-03-11T19:36:19.418+0000 7f839164b700 -1 mds.b *** got signal Terminated *** 2024-03-11T19:36:21.865 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:21 smithi106 podman[73366]: 2024-03-11 19:36:21.567984937 +0000 UTC m=+2.244421092 container died 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-11T19:36:22.146 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:21 smithi106 podman[73366]: 2024-03-11 19:36:21.943488448 +0000 UTC m=+2.619924600 container cleanup 54a3ab9fb018265af830b2da1a757e83ecd2bda344528fa520c1e4b1d33aaf3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-11T19:36:22.146 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 11 19:36:21 smithi106 bash[73366]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-b 2024-03-11T19:36:22.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.b.service' 2024-03-11T19:36:22.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:22.411 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-11T19:36:22.411 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-11T19:36:22.411 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.e 2024-03-11T19:36:22.896 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:22 smithi106 systemd[1]: Stopping Ceph mds.e for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:22.896 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e[65661]: 2024-03-11T19:36:22.721+0000 7f43cbbba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:22.896 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:22 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e[65661]: 2024-03-11T19:36:22.721+0000 7f43cbbba700 -1 mds.e *** got signal Terminated *** 2024-03-11T19:36:25.395 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:25 smithi106 podman[73499]: 2024-03-11 19:36:25.124670505 +0000 UTC m=+2.500270148 container died ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-11T19:36:25.795 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:25 smithi106 podman[73499]: 2024-03-11 19:36:25.439098289 +0000 UTC m=+2.814697932 container cleanup ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:36:25.795 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:25 smithi106 bash[73499]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e 2024-03-11T19:36:25.795 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 11 19:36:25 smithi106 podman[73513]: 2024-03-11 19:36:25.674036824 +0000 UTC m=+0.542343533 container remove ab62c213db272ad1ea477ecd7d6d2e14fa4dcd629236160f887a819191d75b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-11T19:36:25.815 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.e.service' 2024-03-11T19:36:25.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:25.849 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-11T19:36:25.849 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-11T19:36:25.850 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.h 2024-03-11T19:36:26.145 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:25 smithi106 systemd[1]: Stopping Ceph mds.h for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:26.645 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h[65971]: 2024-03-11T19:36:26.183+0000 7fa1ae9d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:26.645 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:26 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h[65971]: 2024-03-11T19:36:26.183+0000 7fa1ae9d6700 -1 mds.h *** got signal Terminated *** 2024-03-11T19:36:29.847 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:29 smithi106 podman[73620]: 2024-03-11 19:36:29.577635622 +0000 UTC m=+3.515590728 container died 344166ecaf739744c9450644da7254cecb7a566337f2e1be788237fde2e862c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:36:30.108 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:29 smithi106 podman[73620]: 2024-03-11 19:36:29.942994837 +0000 UTC m=+3.880949943 container cleanup 344166ecaf739744c9450644da7254cecb7a566337f2e1be788237fde2e862c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:36:30.108 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:29 smithi106 bash[73620]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h 2024-03-11T19:36:30.369 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 11 19:36:30 smithi106 podman[73635]: 2024-03-11 19:36:30.202880737 +0000 UTC m=+0.625699858 container remove 344166ecaf739744c9450644da7254cecb7a566337f2e1be788237fde2e862c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-h, ceph=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:30.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.h.service' 2024-03-11T19:36:30.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:30.408 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-11T19:36:30.408 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-11T19:36:30.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.k 2024-03-11T19:36:30.645 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:30 smithi106 systemd[1]: Stopping Ceph mds.k for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:31.145 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k[65344]: 2024-03-11T19:36:30.738+0000 7f416db80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:31.146 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:30 smithi106 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k[65344]: 2024-03-11T19:36:30.738+0000 7f416db80700 -1 mds.k *** got signal Terminated *** 2024-03-11T19:36:34.244 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:33 smithi106 podman[73741]: 2024-03-11 19:36:33.978813084 +0000 UTC m=+3.362865173 container died faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T19:36:34.586 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:34 smithi106 podman[73741]: 2024-03-11 19:36:34.331356176 +0000 UTC m=+3.715408234 container cleanup faa3487440793eb3ba08bef34922338e3200114f15492ba6c576cd702c60621f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:36:34.586 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 11 19:36:34 smithi106 bash[73741]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-k 2024-03-11T19:36:34.707 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.k.service' 2024-03-11T19:36:34.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:34.743 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-11T19:36:34.743 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-11T19:36:34.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.c 2024-03-11T19:36:35.033 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:34 smithi143 systemd[1]: Stopping Ceph mds.c for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:35.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: 2024-03-11T19:36:35.028+0000 7fbf8bbc3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:35.440 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:35 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c[66817]: 2024-03-11T19:36:35.028+0000 7fbf8bbc3700 -1 mds.c *** got signal Terminated *** 2024-03-11T19:36:37.891 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:37 smithi143 podman[74336]: 2024-03-11 19:36:37.569969183 +0000 UTC m=+2.633542394 container died 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:37.891 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:37 smithi143 podman[74336]: 2024-03-11 19:36:37.887375118 +0000 UTC m=+2.950948317 container cleanup 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-11T19:36:38.190 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:37 smithi143 bash[74336]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c 2024-03-11T19:36:38.190 INFO:journalctl@ceph.mds.c.smithi143.stdout:Mar 11 19:36:38 smithi143 podman[74348]: 2024-03-11 19:36:38.105301332 +0000 UTC m=+0.529818901 container remove 5d919a875e69bb9da583b5f495747988c16f595c39741386b7347d403a1fec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-11T19:36:38.306 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.c.service' 2024-03-11T19:36:38.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:38.344 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-11T19:36:38.345 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-11T19:36:38.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.f 2024-03-11T19:36:38.670 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:38 smithi143 systemd[1]: Stopping Ceph mds.f for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:38.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: 2024-03-11T19:36:38.665+0000 7f1c3fb3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:38.940 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:38 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f[66505]: 2024-03-11T19:36:38.665+0000 7f1c3fb3c700 -1 mds.f *** got signal Terminated *** 2024-03-11T19:36:41.862 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:41 smithi143 podman[74458]: 2024-03-11 19:36:41.570968402 +0000 UTC m=+3.013393732 container died bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-11T19:36:42.190 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:41 smithi143 podman[74458]: 2024-03-11 19:36:41.93301007 +0000 UTC m=+3.375435394 container cleanup bc41a8c6d729a68df1a71b4ccfcde29e789adc5a71b45f8d647a81115e5579e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True) 2024-03-11T19:36:42.287 INFO:journalctl@ceph.mds.f.smithi143.stdout:Mar 11 19:36:41 smithi143 bash[74458]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-f 2024-03-11T19:36:42.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.f.service' 2024-03-11T19:36:42.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:42.387 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-11T19:36:42.387 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-11T19:36:42.387 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.i 2024-03-11T19:36:42.940 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:42 smithi143 systemd[1]: Stopping Ceph mds.i for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:42.940 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: 2024-03-11T19:36:42.685+0000 7f069f6a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:42.940 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:42 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i[67118]: 2024-03-11T19:36:42.685+0000 7f069f6a9700 -1 mds.i *** got signal Terminated *** 2024-03-11T19:36:45.856 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:45 smithi143 podman[74578]: 2024-03-11 19:36:45.576166426 +0000 UTC m=+2.980655891 container died b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-11T19:36:45.857 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:45 smithi143 podman[74578]: 2024-03-11 19:36:45.853069414 +0000 UTC m=+3.257558863 container cleanup b6d009ae86ad9c58f8fc229bdf98a6b5aaf66587ca94915fc8392f510453a80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:36:46.107 INFO:journalctl@ceph.mds.i.smithi143.stdout:Mar 11 19:36:45 smithi143 bash[74578]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-i 2024-03-11T19:36:46.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.i.service' 2024-03-11T19:36:46.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:46.308 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-11T19:36:46.308 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-11T19:36:46.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.l 2024-03-11T19:36:46.690 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:46 smithi143 systemd[1]: Stopping Ceph mds.l for d6472314-dfda-11ee-95c8-87774f69a715... 2024-03-11T19:36:46.690 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l[66181]: 2024-03-11T19:36:46.614+0000 7f56c1c15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:36:46.690 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:46 smithi143 ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l[66181]: 2024-03-11T19:36:46.614+0000 7f56c1c15700 -1 mds.l *** got signal Terminated *** 2024-03-11T19:36:50.911 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:50 smithi143 podman[74699]: 2024-03-11 19:36:50.621547661 +0000 UTC m=+4.097926623 container died 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-11T19:36:51.190 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:50 smithi143 podman[74699]: 2024-03-11 19:36:50.998892309 +0000 UTC m=+4.475271271 container cleanup 9197895d804af680fe8dfbcc892aaa9cf2427df62b8d78bd5bc8b2407af55645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:51.190 INFO:journalctl@ceph.mds.l.smithi143.stdout:Mar 11 19:36:50 smithi143 bash[74699]: ceph-d6472314-dfda-11ee-95c8-87774f69a715-mds-l 2024-03-11T19:36:51.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d6472314-dfda-11ee-95c8-87774f69a715@mds.l.service' 2024-03-11T19:36:51.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:36:51.454 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-11T19:36:51.454 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:36:51.782 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:36:56.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:36:56.590 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:36:59.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:36:59.277 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:37:01.659 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:37:01.762 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:37:01.797 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:37:01.826 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-11T19:37:01.828 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi072/crash 2024-03-11T19:37:01.829 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:37:01.940 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:37:01.940 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:37:01.945 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi106/crash 2024-03-11T19:37:01.946 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:37:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:37:01.997 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:37:01.998 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi143/crash 2024-03-11T19:37:01.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:37:02.032 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/d6472314-dfda-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:37:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:37:02.034 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-11T19:37:02.034 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T19:37:02.134 INFO:teuthology.orchestra.run.smithi072.stdout:2024-03-11T19:12:38.883253+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:37:02.135 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-11T19:37:02.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[SEC\]' /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T19:37:02.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]' /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T19:37:02.302 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[WRN\]' /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T19:37:02.378 INFO:teuthology.orchestra.run.smithi072.stdout:2024-03-11T19:12:38.883253+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:37:02.379 INFO:tasks.cephadm:Compressing logs... 2024-03-11T19:37:02.380 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:37:02.384 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:37:02.386 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:37:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T19:37:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:: No such file or directory 2024-03-11T19:37:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.l.log 2024-03-11T19:37:02.422 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.i.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.11.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.z.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.i.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.8.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.z.log: gzip -5 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.5.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.f.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.f.log: 90.1% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.2.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log: 94.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.l.log.gz 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.c.log 2024-03-11T19:37:02.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.2.log: 87.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:37:02.424 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T19:37:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.z.log.gz 2024-03-11T19:37:02.427 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.c.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:: No such file or directory 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.10.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.b.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.4.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.1.log 2024-03-11T19:37:02.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.k.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.h.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.1.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.7.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.e.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.y.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.k.log.gz 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.y.log: 94.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.e.log.gz 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:37:02.432 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:37:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T19:37:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:37:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.b.log 2024-03-11T19:37:02.433 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.y.log.gz 2024-03-11T19:37:02.438 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:37:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:37:02.448 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-11T19:37:02.448 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T19:37:02.449 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.6.log 2024-03-11T19:37:02.450 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.a.log 2024-03-11T19:37:02.451 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:37:02.452 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.g.log 2024-03-11T19:37:02.454 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.j.log 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.3.log 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.x.log 2024-03-11T19:37:02.458 INFO:teuthology.orchestra.run.smithi072.stderr: 93.2% 89.9% -- replaced with /var/log/ceph/cephadm.log.gz -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:37:02.458 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-03-11T19:37:02.458 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.0.log 2024-03-11T19:37:02.461 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:37:02.463 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.0.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:37:02.464 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:37:02.464 INFO:teuthology.orchestra.run.smithi072.stderr: 94.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.j.log.gz 2024-03-11T19:37:02.464 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T19:37:02.468 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.9.log 2024-03-11T19:37:02.469 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:37:02.470 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.d.log 2024-03-11T19:37:02.471 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.a.log 2024-03-11T19:37:02.476 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.a.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.d.log: /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-11T19:37:02.486 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.log: 94.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.d.log.gz 2024-03-11T19:37:02.486 INFO:teuthology.orchestra.run.smithi072.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-11T19:37:02.488 INFO:teuthology.orchestra.run.smithi072.stderr: 90.9% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:37:02.489 INFO:teuthology.orchestra.run.smithi143.stderr: 94.6% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.c.log.gz 2024-03-11T19:37:02.492 INFO:teuthology.orchestra.run.smithi072.stderr: 94.4% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.a.log.gz 2024-03-11T19:37:02.499 INFO:teuthology.orchestra.run.smithi106.stderr: 93.8% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.h.log.gz 2024-03-11T19:37:02.537 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.g.log.gz 2024-03-11T19:37:02.623 INFO:teuthology.orchestra.run.smithi106.stderr: 93.2% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.b.log.gz 2024-03-11T19:37:02.789 INFO:teuthology.orchestra.run.smithi143.stderr: 91.8% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.c.log.gz 2024-03-11T19:37:02.824 INFO:teuthology.orchestra.run.smithi072.stderr: 89.8% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mgr.x.log.gz 2024-03-11T19:37:02.858 INFO:teuthology.orchestra.run.smithi143.stderr: 95.8% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.f.log.gz 2024-03-11T19:37:02.926 INFO:teuthology.orchestra.run.smithi106.stderr: 91.8% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.b.log.gz 2024-03-11T19:37:03.462 INFO:teuthology.orchestra.run.smithi072.stderr: 90.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mon.a.log.gz 2024-03-11T19:37:04.155 INFO:teuthology.orchestra.run.smithi106.stderr: 92.2% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.10.log.gz 2024-03-11T19:37:04.671 INFO:teuthology.orchestra.run.smithi106.stderr: 92.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.1.log.gz 2024-03-11T19:37:04.697 INFO:teuthology.orchestra.run.smithi072.stderr: 91.9% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.9.log.gz 2024-03-11T19:37:04.752 INFO:teuthology.orchestra.run.smithi072.stderr: 92.2% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.0.log.gz 2024-03-11T19:37:04.953 INFO:teuthology.orchestra.run.smithi072.stderr: 92.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.6.log.gz 2024-03-11T19:37:05.080 INFO:teuthology.orchestra.run.smithi106.stderr: 92.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.7.log.gz 2024-03-11T19:37:05.181 INFO:teuthology.orchestra.run.smithi143.stderr: 92.7% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.11.log.gz 2024-03-11T19:37:05.196 INFO:teuthology.orchestra.run.smithi143.stderr: 93.1% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.2.log.gz 2024-03-11T19:37:05.444 INFO:teuthology.orchestra.run.smithi106.stderr: 92.5% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.4.log.gz 2024-03-11T19:37:05.445 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-03-11T19:37:05.445 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m3.045s 2024-03-11T19:37:05.445 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m10.157s 2024-03-11T19:37:05.445 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.370s 2024-03-11T19:37:05.451 INFO:teuthology.orchestra.run.smithi072.stderr: 92.2% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.3.log.gz 2024-03-11T19:37:05.456 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-03-11T19:37:05.456 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m3.041s 2024-03-11T19:37:05.456 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m10.709s 2024-03-11T19:37:05.456 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.912s 2024-03-11T19:37:05.496 INFO:teuthology.orchestra.run.smithi143.stderr: 93.3% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-mds.i.log.gz 2024-03-11T19:37:05.547 INFO:teuthology.orchestra.run.smithi143.stderr: 92.6% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.5.log.gz 2024-03-11T19:37:06.796 INFO:teuthology.orchestra.run.smithi143.stderr: 94.7% -- replaced with /var/log/ceph/d6472314-dfda-11ee-95c8-87774f69a715/ceph-osd.8.log.gz 2024-03-11T19:37:06.798 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-03-11T19:37:06.798 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m4.398s 2024-03-11T19:37:06.798 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m16.442s 2024-03-11T19:37:06.798 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m0.590s 2024-03-11T19:37:06.798 INFO:tasks.cephadm:Archiving logs... 2024-03-11T19:37:06.799 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi072/log 2024-03-11T19:37:06.800 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:37:08.635 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi106/log 2024-03-11T19:37:08.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:37:10.276 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi143/log 2024-03-11T19:37:10.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:37:12.549 INFO:tasks.cephadm:Removing cluster... 2024-03-11T19:37:12.550 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force 2024-03-11T19:37:12.879 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:37:13.716 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force 2024-03-11T19:37:13.905 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:37:14.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d6472314-dfda-11ee-95c8-87774f69a715 --force 2024-03-11T19:37:14.360 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: d6472314-dfda-11ee-95c8-87774f69a715 2024-03-11T19:37:14.631 INFO:tasks.cephadm:Removing cephadm ... 2024-03-11T19:37:14.631 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:37:14.690 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:37:14.706 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:37:14.721 INFO:tasks.cephadm:Teardown complete 2024-03-11T19:37:14.722 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-11T19:37:14.733 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-11T19:37:14.734 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:37:14.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:37:14.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:37:14.906 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-11T19:37:14.920 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-11T19:37:14.920 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:14.929 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:14.931 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:14.944 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:14.951 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:14.951 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-03-11T19:37:14.951 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +1071us[+1193us] +/- 131ms 2024-03-11T19:37:14.951 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -2169us[-2047us] +/- 64ms 2024-03-11T19:37:14.952 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +1893us[+2015us] +/- 68ms 2024-03-11T19:37:14.952 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +1134us[+1134us] +/- 131ms 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv02.front.sepia.ceph.com 3 7 377 81 -2085us[-2154us] +/- 63ms 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1919us[+1919us] +/- 68ms 2024-03-11T19:37:14.953 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:14.974 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:14.985 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:14.986 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-03-11T19:37:14.986 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +1087us[+1087us] +/- 131ms 2024-03-11T19:37:14.986 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv02.front.sepia.ceph.com 3 8 377 141 -2138us[-2150us] +/- 62ms 2024-03-11T19:37:14.986 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1834us[+1834us] +/- 68ms 2024-03-11T19:37:14.986 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:14.987 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-11T19:37:14.997 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-11T19:37:14.998 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-11T19:37:15.022 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:37:15.260 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:37:15.339 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:37:15.410 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:37:15.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:15.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:37:15.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:15.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:37:15.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:15.575 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-11T19:37:15.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-11T19:37:15.606 INFO:teuthology.task.internal:Duration was 2227.846561 seconds 2024-03-11T19:37:15.607 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-11T19:37:15.639 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-11T19:37:15.640 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:37:15.648 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:37:15.650 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:37:15.693 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:37:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:37:15.798 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:37:16.258 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-11T19:37:16.258 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-03-11T19:37:16.258 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:37:16.319 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-03-11T19:37:16.319 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:37:16.346 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-03-11T19:37:16.346 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:37:16.380 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-11T19:37:16.380 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:37:16.390 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:37:16.393 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:37:16.475 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-11T19:37:16.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:37:16.481 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:37:16.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:37:16.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-11T19:37:16.998 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-11T19:37:16.998 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:37:17.095 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:37:17.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:37:17.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-11T19:37:17.162 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.170 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.195 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-03-11T19:37:17.216 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-03-11T19:37:17.272 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-03-11T19:37:17.324 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:17.376 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:17.390 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:37:17.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:37:17.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-11T19:37:17.416 INFO:teuthology.task.internal:Transferring archived files... 2024-03-11T19:37:17.417 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi072 2024-03-11T19:37:17.418 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:37:17.530 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi106 2024-03-11T19:37:17.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:37:17.569 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259/remote/smithi143 2024-03-11T19:37:17.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:37:17.607 INFO:teuthology.task.internal:Removing archive directory... 2024-03-11T19:37:17.608 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:37:17.616 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:37:17.618 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:37:17.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-11T19:37:17.679 INFO:teuthology.task.internal:Not uploading archives. 2024-03-11T19:37:17.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-11T19:37:17.742 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-11T19:37:17.743 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:37:17.750 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:37:17.752 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:37:17.764 INFO:teuthology.orchestra.run.smithi106.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:37 /home/ubuntu/cephtest 2024-03-11T19:37:17.766 INFO:teuthology.orchestra.run.smithi143.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:37 /home/ubuntu/cephtest 2024-03-11T19:37:17.800 INFO:teuthology.orchestra.run.smithi072.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:37 /home/ubuntu/cephtest 2024-03-11T19:37:17.803 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-11T19:37:17.815 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-11T19:37:17.882 INFO:teuthology.nuke:Checking targets against current locks 2024-03-11T19:37:17.908 DEBUG:teuthology.nuke:shortname: smithi072 2024-03-11T19:37:17.909 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:37:17.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.573305', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVqXqWtJm6fZ0pnTaM1i7DIeC4Ww+DgoOBY7RfyaD6rQf2472fJIfu/PWrQTXXT0O5OYc0TqExv2cqAoTYzqHY='} 2024-03-11T19:37:17.938 DEBUG:teuthology.nuke:shortname: smithi106 2024-03-11T19:37:17.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:37:17.961 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.574642', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOc0GKDBSBvCA0swtD/8Mb7zanz47X4xx7vSMex+ndErKYfZQACUTyvU9TlhmL3bA95OTI3JkFpUUpBHIgPpSQw='} 2024-03-11T19:37:17.969 DEBUG:teuthology.nuke:shortname: smithi143 2024-03-11T19:37:17.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:37:17.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:49:41.571727', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE+UwAq37y0s5vpPONSzDIQm2UPZ6ctj6LvDzxHcSvsZGm9kbLzQx9eizAdRULCRXTQ0qYQiZRC91LVDv2g7zRs='} 2024-03-11T19:37:18.002 INFO:teuthology.orchestra.console.smithi072:Power off 2024-03-11T19:37:18.003 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:37:18.027 INFO:teuthology.orchestra.console.smithi106:Power off 2024-03-11T19:37:18.028 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:37:18.049 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-03-11T19:37:18.049 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:18.078 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-03-11T19:37:18.078 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:18.100 INFO:teuthology.orchestra.console.smithi143:Power off 2024-03-11T19:37:18.100 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:37:18.110 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-03-11T19:37:18.112 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-03-11T19:37:18.136 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2024-03-11T19:37:18.136 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:18.164 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-11T19:37:22.111 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:22.121 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:22.164 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:22.224 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-03-11T19:37:22.231 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-03-11T19:37:22.276 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-11T19:37:26.229 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:26.239 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:26.277 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:26.342 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-03-11T19:37:26.351 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-03-11T19:37:26.393 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-11T19:37:26.443 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-03-11T19:37:26.452 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-03-11T19:37:30.394 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:37:30.505 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-03-11T19:37:30.606 INFO:teuthology.orchestra.console.smithi143:Power off completed 2024-03-11T19:37:30.711 INFO:teuthology.run:Summary data: client.0-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} duration: 2227.8465609550476 failure_reason: '"2024-03-11T19:12:38.883253+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-11T19:37:30.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T19:37:30.820 INFO:teuthology.run:FAIL